RPTools.net

Discussion and Support

Skip to content

It is currently Mon Dec 18, 2017 1:17 pm 






Reply to topic  [ 64 posts ]  Go to page 1, 2, 3, 4, 5  Next

Previous topic | Next topic 

  Print view

Author Message
User avatar  Offline
Dragon
 
Joined: Fri Mar 23, 2007 10:27 am
Posts: 778
Location: Sussex, UK.
 Post subject: DDi Compendium API and Java Classes?
PostPosted: Mon Jun 14, 2010 5:34 am 
Does anyone know if anyone has written a Java class to grab stuff from the DDi API? I'm specifically after grabbing monsters from the Compendium and don't want to write my own class if someone has already done all the work for me...

Cheers
Blakey

_________________
The guy in the green hat.


Top
 Profile  
 
 Offline
Great Wyrm
 
Joined: Tue May 26, 2009 7:51 pm
Posts: 2060
Location: Philadelphia, PA
 Post subject: Re: DDi Compendium API and Java Classes?
PostPosted: Mon Jun 14, 2010 8:33 am 
I know VCC uses JAVA and pulls the Monsters out of the compendium using a Firefox extension.


Top
 Profile  
 
User avatar  Offline
Dragon
 
Joined: Fri Mar 23, 2007 10:27 am
Posts: 778
Location: Sussex, UK.
 Post subject: Re: DDi Compendium API and Java Classes?
PostPosted: Mon Jun 14, 2010 9:15 am 
Any clues as to what VCC is? I looked in google and found:

VCC - Wikipedia, the free encyclopedia VCC is a three-letter acronym that may mean: Common-collector voltage (VCC)
The Veteran Car Club of Great Britain
The Veteran Cycle Club (V-CC) home page
Vancouver Community College - B.C.'s No. 1 College
Voice for the Child in Care (VCC)

None of these seemed likely to have written a DDi Java interface... :D

_________________
The guy in the green hat.


Top
 Profile  
 
 Offline
Great Wyrm
 
Joined: Tue May 26, 2009 7:51 pm
Posts: 2060
Location: Philadelphia, PA
 Post subject: Re: DDi Compendium API and Java Classes?
PostPosted: Mon Jun 14, 2010 10:57 am 
Here it is: VCC


Top
 Profile  
 
User avatar  Offline
Giant
 
Joined: Sun Jun 22, 2008 12:23 am
Posts: 231
Location: Tampa, FL
 Post subject: Re: DDi Compendium API and Java Classes?
PostPosted: Tue Jun 15, 2010 1:00 pm 
That sounds very interesting. It would be awesome if we could use the DDI content to easily create monster tokens and such. There is a thread about parsing a PDF and also DDI text and creating a sheet from that http://forums.rptools.net/viewtopic.php?f=8&t=13237. So I would imagine some of the heavy lifting is already done.

The fella at the VCC project indicates that he wrote an extension for firefox that yanks the data from DDI. I would imagine that code is available. A little bit of cutting and pasting and you may be closer than you think...

_________________
When the boogeyman goes to bed everynight, he checks his closet for Chuck Norris.


Top
 Profile  
 
User avatar  Offline
Dragon
 
Joined: Fri Mar 23, 2007 10:27 am
Posts: 778
Location: Sussex, UK.
 Post subject: Re: DDi Compendium API and Java Classes?
PostPosted: Wed Jun 16, 2010 3:48 am 
I'm doing this more as an intellectual exercise and for practicing Java than anything else, so I'll probably not be stealing other people's code, even though I could. I might steal it for peeking at though and seeing how they do stuff.

I've so far figured out how to run a query on the compendium to get an XML file with a set of results in it - so I can ask it to give me a Goblin Blackblade and I'll get an XML file back which has the monster's ID number. This can then be sent as a query to get back the HTML page for that particular monster. At the moment this doesn't quite work as I'm hitting a wall trying to get past the DDi authentication. I'm currently looking into how to login programatically to get around this.

Once that is done I should be able to tell my java class 'get me a goblin blackblade' and have it return the HTML code for the monster. I'll then have to parse that into something which makes sense to MapTool but that is another set of issues entirely as I have to start thinking about frameworks at that stage.

Simultaneously with this, as part of the same package, I'm writing a java parser to parse the Character Builder save files and build a PC object. That is coming along nicely too, although there is a problem getting the details of the actual powers as these are not kept in the save file. But I'm thinking I might be able to pop off to the Compendium and grab the HTML for the power once I have the power name. We shall see.

Maybe I'll make a new thread where I can track my progress. Would anyone be interested in that?

Cheers
Blakey

P.S. This will be slow and steady work and I'm liable to just stop working on it at any moment so don't expect anything from me on this effort at all!! :D

_________________
The guy in the green hat.


Top
 Profile  
 
User avatar  Offline
Dragon
 
Joined: Fri Mar 23, 2007 10:27 am
Posts: 778
Location: Sussex, UK.
 Post subject: Re: DDi Compendium API and Java Classes?
PostPosted: Wed Jun 16, 2010 4:13 am 
On this subject, can anyone figure out how to effectively pass your login details to the compendium on the URL line of a browser? I have figured this much so far:

If I want to go to a given power (say Divine Armor, which is power number 959), and also login in the same breath, I use this URL:

Code:
http://www.wizards.com/dndinsider/compendium/login.aspx?page=power&id=959


That will take me to the login page and then when I have logged in, it will forward me to the power page and select power 959. However I really want to pass my login and password details in here too, so that it takes me straight to the page without asking for any input, regardless of whether you are logged in or not - effectively guaranteeing a login as it goes. Something like this:

Code:
http://www.wizards.com/dndinsider/compendium/login.aspx?email=my.email@hotmail.com&password=myPassWord&page=power&id=959


I just don't know what varibales I need to set up to pass through to the login page automatically. Can anyone help?

Cheers
Blakey

_________________
The guy in the green hat.


Top
 Profile  
 
User avatar  Offline
Deity
 
Joined: Tue Sep 11, 2007 6:31 pm
Posts: 5412
 Post subject: Re: DDi Compendium API and Java Classes?
PostPosted: Wed Jun 16, 2010 8:04 am 
The way many websites work is by passing a cookie back in the response. You then have to pass that cookie back in your next request. Flow is something like this:
  1. Open connection to Site, passing in username/password(typically to the login page URL of the site)
  2. Get the cookie data from the response object passed back from the site.
  3. Store the cookie data in a global value for future use
  4. create a new connection to the next page you wish to go to, passing the cookie data in as part of your request headers.
  5. Repeat step 4 for each page you wish to access.

Google "java pass cookie to URL"

_________________
I save all my Campaign Files to DropBox. Not only can I access a campaign file from pretty much any OS that will run Maptool(Win,OSX, linux), but each file is versioned, so if something goes crazy wild, I can always roll back to a previous version of the same file.

Get your Dropbox 2GB via my referral link, and as a bonus, I get an extra 250 MB of space. Even if you don't don't use my link, I still enthusiastically recommend Dropbox..


Top
 Profile  
 
User avatar  Offline
Dragon
 
Joined: Fri Mar 23, 2007 10:27 am
Posts: 778
Location: Sussex, UK.
 Post subject: Re: DDi Compendium API and Java Classes?
PostPosted: Wed Jun 16, 2010 8:24 am 
jfrazierjr wrote:
The way many websites work is by passing a cookie back in the response. You then have to pass that cookie back in your next request. Flow is something like this:
  1. Open connection to Site, passing in username/password(typically to the login page URL of the site)
  2. Get the cookie data from the response object passed back from the site.
  3. Store the cookie data in a global value for future use
  4. create a new connection to the next page you wish to go to, passing the cookie data in as part of your request headers.
  5. Repeat step 4 for each page you wish to access.

Google "java pass cookie to URL"


Thanks.

That makes sense but unless I'm missing something, it is step 1 that I don't know how to do. How do you login to the web page (www.wizards.com/.../login.aspx) with your username and password unless you use "www.wizards.com/.../login.aspx?email=a@b.c&password=pw" - and I don't know what parameters to pass in here as I can't get this to work by hand.

Or am I getting more confused than I think I am?

Cheers for the help!

_________________
The guy in the green hat.


Top
 Profile  
 
User avatar  Offline
Great Wyrm
 
Joined: Wed Nov 15, 2006 9:03 am
Posts: 1023
 Post subject: Re: DDi Compendium API and Java Classes?
PostPosted: Wed Jun 16, 2010 8:31 am 
I think you're confusing yourself. What i understand JF to be saying is basically, manually login to the site, and get the cookie (mmm, cooooookkkkieeeeee) that that login generates.


Top
 Profile  
 
 Offline
Dragon
 
Joined: Tue Mar 24, 2009 2:34 pm
Posts: 859
 Post subject: Re: DDi Compendium API and Java Classes?
PostPosted: Wed Jun 16, 2010 8:41 am 
I have some perl code that scrapes the Compendium. The sign in process uses POST rather than GET, so you can't pass the parameters via the URL. I don't know how you'd send a POST request in Java, but I'm sure there's a way. :)

_________________
Reed (halfling sorcerer P3) // Collin (human fighter P2) // Cine (eladrin psion H2)
Sirath (deva shaman H1) // Alesk (dragonborn cleric H3) // Kevin (halfling barbarian H1)


Top
 Profile  
 
User avatar  Offline
Dragon
 
Joined: Fri Mar 23, 2007 10:27 am
Posts: 778
Location: Sussex, UK.
 Post subject: Re: DDi Compendium API and Java Classes?
PostPosted: Wed Jun 16, 2010 8:50 am 
aku wrote:
I think you're confusing yourself. What i understand JF to be saying is basically, manually login to the site, and get the cookie (mmm, cooooookkkkieeeeee) that that login generates.


Err, I can login but how do I generate a cookie from that? Sorry for the stupid question!!!

_________________
The guy in the green hat.


Top
 Profile  
 
User avatar  Offline
Deity
 
Joined: Tue Sep 11, 2007 6:31 pm
Posts: 5412
 Post subject: Re: DDi Compendium API and Java Classes?
PostPosted: Wed Jun 16, 2010 8:52 am 
Blakey wrote:
jfrazierjr wrote:
The way many websites work is by passing a cookie back in the response. You then have to pass that cookie back in your next request. Flow is something like this:
  1. Open connection to Site, passing in username/password(typically to the login page URL of the site)
  2. Get the cookie data from the response object passed back from the site.
  3. Store the cookie data in a global value for future use
  4. create a new connection to the next page you wish to go to, passing the cookie data in as part of your request headers.
  5. Repeat step 4 for each page you wish to access.

Google "java pass cookie to URL"


Thanks.

That makes sense but unless I'm missing something, it is step 1 that I don't know how to do. How do you login to the web page (http://www.wizards.com/.../login.aspx) with your username and password unless you use "www.wizards.com/.../login.aspx?email=a@b.c&password=pw" - and I don't know what parameters to pass in here as I can't get this to work by hand.

Or am I getting more confused than I think I am?

Cheers for the help!


Ok... First, you have to go to the login page in the site and view the html source to find the username/password(which may not be called that!!!!!) fields as well as the enclosing <form> element(so you can get the action and method). In many cases, sites may ONLY allow credentials to be passed in via POST method and not GET (thats what you are doing when you add the things to the end of a URL).

Once you have that information, you can then begin your code to login programmaticlly.

_________________
I save all my Campaign Files to DropBox. Not only can I access a campaign file from pretty much any OS that will run Maptool(Win,OSX, linux), but each file is versioned, so if something goes crazy wild, I can always roll back to a previous version of the same file.

Get your Dropbox 2GB via my referral link, and as a bonus, I get an extra 250 MB of space. Even if you don't don't use my link, I still enthusiastically recommend Dropbox..


Top
 Profile  
 
User avatar  Offline
Dragon
 
Joined: Fri Mar 23, 2007 10:27 am
Posts: 778
Location: Sussex, UK.
 Post subject: Re: DDi Compendium API and Java Classes?
PostPosted: Wed Jun 16, 2010 9:07 am 
Thanlis wrote:
I have some perl code that scrapes the Compendium. The sign in process uses POST rather than GET, so you can't pass the parameters via the URL. I don't know how you'd send a POST request in Java, but I'm sure there's a way. :)


I can easily figure out how to do a POST in java. The question is what do I need to post?

The login.aspx file labels the two fields 'email' and 'password'. Is that all I need to pass in?

Cheers
Blakey

_________________
The guy in the green hat.


Top
 Profile  
 
User avatar  Offline
Deity
 
Joined: Tue Sep 11, 2007 6:31 pm
Posts: 5412
 Post subject: Re: DDi Compendium API and Java Classes?
PostPosted: Wed Jun 16, 2010 9:27 am 
Blakey wrote:
Thanlis wrote:
I have some perl code that scrapes the Compendium. The sign in process uses POST rather than GET, so you can't pass the parameters via the URL. I don't know how you'd send a POST request in Java, but I'm sure there's a way. :)


I can easily figure out how to do a POST in java. The question is what do I need to post?

The login.aspx file labels the two fields 'email' and 'password'. Is that all I need to pass in?

Cheers
Blakey

You should pass EVERY form element on the page. Example:
Code:
<input type="submit" name="InsiderSignin" value="Sign In" id="InsiderSignin" />

should be: InsiderSignin=Sign In


The BEST way to do this is to use Java to do a GET on the login page itself. Then in the response document, interate though the page's form elements and grab each <input> tags name and value. Then, post those name/value pairs to the login.aspx page. When you get the response from this page, pull the cookie values out of the header and store for later use.

_________________
I save all my Campaign Files to DropBox. Not only can I access a campaign file from pretty much any OS that will run Maptool(Win,OSX, linux), but each file is versioned, so if something goes crazy wild, I can always roll back to a previous version of the same file.

Get your Dropbox 2GB via my referral link, and as a bonus, I get an extra 250 MB of space. Even if you don't don't use my link, I still enthusiastically recommend Dropbox..


Top
 Profile  
 
Display posts from previous:  Sort by  
Reply to topic  [ 64 posts ]  Go to page 1, 2, 3, 4, 5  Next

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:

Who is online

In total there is 1 user online :: 0 registered, 0 hidden and 1 guest (based on users active over the past 5 minutes)
Most users ever online was 243 on Sun Nov 04, 2012 6:14 am

Users browsing this forum: No registered users and 1 guest





Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group

Style based on Andreas08 by Andreas Viklund

Style by Elizabeth Shulman