DDi Compendium API and Java Classes?
Moderators: dorpond, trevor, Azhrei
DDi Compendium API and Java Classes?
Does anyone know if anyone has written a Java class to grab stuff from the DDi API? I'm specifically after grabbing monsters from the Compendium and don't want to write my own class if someone has already done all the work for me...
Cheers
Blakey
Cheers
Blakey
The guy in the green hat.
Re: DDi Compendium API and Java Classes?
I know VCC uses JAVA and pulls the Monsters out of the compendium using a Firefox extension.
Re: DDi Compendium API and Java Classes?
Any clues as to what VCC is? I looked in google and found:
VCC - Wikipedia, the free encyclopedia VCC is a three-letter acronym that may mean: Common-collector voltage (VCC)
The Veteran Car Club of Great Britain
The Veteran Cycle Club (V-CC) home page
Vancouver Community College - B.C.'s No. 1 College
Voice for the Child in Care (VCC)
None of these seemed likely to have written a DDi Java interface...
VCC - Wikipedia, the free encyclopedia VCC is a three-letter acronym that may mean: Common-collector voltage (VCC)
The Veteran Car Club of Great Britain
The Veteran Cycle Club (V-CC) home page
Vancouver Community College - B.C.'s No. 1 College
Voice for the Child in Care (VCC)
None of these seemed likely to have written a DDi Java interface...
The guy in the green hat.
Re: DDi Compendium API and Java Classes?
Here it is: VCC
Re: DDi Compendium API and Java Classes?
That sounds very interesting. It would be awesome if we could use the DDI content to easily create monster tokens and such. There is a thread about parsing a PDF and also DDI text and creating a sheet from that http://forums.rptools.net/viewtopic.php?f=8&t=13237. So I would imagine some of the heavy lifting is already done.
The fella at the VCC project indicates that he wrote an extension for firefox that yanks the data from DDI. I would imagine that code is available. A little bit of cutting and pasting and you may be closer than you think...
The fella at the VCC project indicates that he wrote an extension for firefox that yanks the data from DDI. I would imagine that code is available. A little bit of cutting and pasting and you may be closer than you think...
When the boogeyman goes to bed everynight, he checks his closet for Chuck Norris.
Re: DDi Compendium API and Java Classes?
I'm doing this more as an intellectual exercise and for practicing Java than anything else, so I'll probably not be stealing other people's code, even though I could. I might steal it for peeking at though and seeing how they do stuff.
I've so far figured out how to run a query on the compendium to get an XML file with a set of results in it - so I can ask it to give me a Goblin Blackblade and I'll get an XML file back which has the monster's ID number. This can then be sent as a query to get back the HTML page for that particular monster. At the moment this doesn't quite work as I'm hitting a wall trying to get past the DDi authentication. I'm currently looking into how to login programatically to get around this.
Once that is done I should be able to tell my java class 'get me a goblin blackblade' and have it return the HTML code for the monster. I'll then have to parse that into something which makes sense to MapTool but that is another set of issues entirely as I have to start thinking about frameworks at that stage.
Simultaneously with this, as part of the same package, I'm writing a java parser to parse the Character Builder save files and build a PC object. That is coming along nicely too, although there is a problem getting the details of the actual powers as these are not kept in the save file. But I'm thinking I might be able to pop off to the Compendium and grab the HTML for the power once I have the power name. We shall see.
Maybe I'll make a new thread where I can track my progress. Would anyone be interested in that?
Cheers
Blakey
P.S. This will be slow and steady work and I'm liable to just stop working on it at any moment so don't expect anything from me on this effort at all!!
I've so far figured out how to run a query on the compendium to get an XML file with a set of results in it - so I can ask it to give me a Goblin Blackblade and I'll get an XML file back which has the monster's ID number. This can then be sent as a query to get back the HTML page for that particular monster. At the moment this doesn't quite work as I'm hitting a wall trying to get past the DDi authentication. I'm currently looking into how to login programatically to get around this.
Once that is done I should be able to tell my java class 'get me a goblin blackblade' and have it return the HTML code for the monster. I'll then have to parse that into something which makes sense to MapTool but that is another set of issues entirely as I have to start thinking about frameworks at that stage.
Simultaneously with this, as part of the same package, I'm writing a java parser to parse the Character Builder save files and build a PC object. That is coming along nicely too, although there is a problem getting the details of the actual powers as these are not kept in the save file. But I'm thinking I might be able to pop off to the Compendium and grab the HTML for the power once I have the power name. We shall see.
Maybe I'll make a new thread where I can track my progress. Would anyone be interested in that?
Cheers
Blakey
P.S. This will be slow and steady work and I'm liable to just stop working on it at any moment so don't expect anything from me on this effort at all!!
The guy in the green hat.
Re: DDi Compendium API and Java Classes?
On this subject, can anyone figure out how to effectively pass your login details to the compendium on the URL line of a browser? I have figured this much so far:
If I want to go to a given power (say Divine Armor, which is power number 959), and also login in the same breath, I use this URL:
That will take me to the login page and then when I have logged in, it will forward me to the power page and select power 959. However I really want to pass my login and password details in here too, so that it takes me straight to the page without asking for any input, regardless of whether you are logged in or not - effectively guaranteeing a login as it goes. Something like this:
I just don't know what varibales I need to set up to pass through to the login page automatically. Can anyone help?
Cheers
Blakey
If I want to go to a given power (say Divine Armor, which is power number 959), and also login in the same breath, I use this URL:
Code: Select all
http://www.wizards.com/dndinsider/compendium/login.aspx?page=power&id=959
Code: Select all
http://www.wizards.com/dndinsider/compendium/[email protected]&password=myPassWord&page=power&id=959
Cheers
Blakey
The guy in the green hat.
- jfrazierjr
- Deity
- Posts: 5176
- Joined: Tue Sep 11, 2007 7:31 pm
Re: DDi Compendium API and Java Classes?
The way many websites work is by passing a cookie back in the response. You then have to pass that cookie back in your next request. Flow is something like this:
- Open connection to Site, passing in username/password(typically to the login page URL of the site)
- Get the cookie data from the response object passed back from the site.
- Store the cookie data in a global value for future use
- create a new connection to the next page you wish to go to, passing the cookie data in as part of your request headers.
- Repeat step 4 for each page you wish to access.
I save all my Campaign Files to DropBox. Not only can I access a campaign file from pretty much any OS that will run Maptool(Win,OSX, linux), but each file is versioned, so if something goes crazy wild, I can always roll back to a previous version of the same file.
Get your Dropbox 2GB via my referral link, and as a bonus, I get an extra 250 MB of space. Even if you don't don't use my link, I still enthusiastically recommend Dropbox..
Get your Dropbox 2GB via my referral link, and as a bonus, I get an extra 250 MB of space. Even if you don't don't use my link, I still enthusiastically recommend Dropbox..
Re: DDi Compendium API and Java Classes?
Thanks.jfrazierjr wrote:The way many websites work is by passing a cookie back in the response. You then have to pass that cookie back in your next request. Flow is something like this:Google "java pass cookie to URL"
- Open connection to Site, passing in username/password(typically to the login page URL of the site)
- Get the cookie data from the response object passed back from the site.
- Store the cookie data in a global value for future use
- create a new connection to the next page you wish to go to, passing the cookie data in as part of your request headers.
- Repeat step 4 for each page you wish to access.
That makes sense but unless I'm missing something, it is step 1 that I don't know how to do. How do you login to the web page (www.wizards.com/.../login.aspx) with your username and password unless you use "www.wizards.com/.../[email protected]&password=pw" - and I don't know what parameters to pass in here as I can't get this to work by hand.
Or am I getting more confused than I think I am?
Cheers for the help!
The guy in the green hat.
Re: DDi Compendium API and Java Classes?
I think you're confusing yourself. What i understand JF to be saying is basically, manually login to the site, and get the cookie (mmm, cooooookkkkieeeeee) that that login generates.
Re: DDi Compendium API and Java Classes?
I have some perl code that scrapes the Compendium. The sign in process uses POST rather than GET, so you can't pass the parameters via the URL. I don't know how you'd send a POST request in Java, but I'm sure there's a way.
Re: DDi Compendium API and Java Classes?
Err, I can login but how do I generate a cookie from that? Sorry for the stupid question!!!aku wrote:I think you're confusing yourself. What i understand JF to be saying is basically, manually login to the site, and get the cookie (mmm, cooooookkkkieeeeee) that that login generates.
The guy in the green hat.
- jfrazierjr
- Deity
- Posts: 5176
- Joined: Tue Sep 11, 2007 7:31 pm
Re: DDi Compendium API and Java Classes?
Ok... First, you have to go to the login page in the site and view the html source to find the username/password(which may not be called that!!!!!) fields as well as the enclosing <form> element(so you can get the action and method). In many cases, sites may ONLY allow credentials to be passed in via POST method and not GET (thats what you are doing when you add the things to the end of a URL).Blakey wrote:Thanks.jfrazierjr wrote:The way many websites work is by passing a cookie back in the response. You then have to pass that cookie back in your next request. Flow is something like this:Google "java pass cookie to URL"
- Open connection to Site, passing in username/password(typically to the login page URL of the site)
- Get the cookie data from the response object passed back from the site.
- Store the cookie data in a global value for future use
- create a new connection to the next page you wish to go to, passing the cookie data in as part of your request headers.
- Repeat step 4 for each page you wish to access.
That makes sense but unless I'm missing something, it is step 1 that I don't know how to do. How do you login to the web page (http://www.wizards.com/.../login.aspx) with your username and password unless you use "www.wizards.com/.../[email protected]&password=pw" - and I don't know what parameters to pass in here as I can't get this to work by hand.
Or am I getting more confused than I think I am?
Cheers for the help!
Once you have that information, you can then begin your code to login programmaticlly.
I save all my Campaign Files to DropBox. Not only can I access a campaign file from pretty much any OS that will run Maptool(Win,OSX, linux), but each file is versioned, so if something goes crazy wild, I can always roll back to a previous version of the same file.
Get your Dropbox 2GB via my referral link, and as a bonus, I get an extra 250 MB of space. Even if you don't don't use my link, I still enthusiastically recommend Dropbox..
Get your Dropbox 2GB via my referral link, and as a bonus, I get an extra 250 MB of space. Even if you don't don't use my link, I still enthusiastically recommend Dropbox..
Re: DDi Compendium API and Java Classes?
I can easily figure out how to do a POST in java. The question is what do I need to post?Thanlis wrote:I have some perl code that scrapes the Compendium. The sign in process uses POST rather than GET, so you can't pass the parameters via the URL. I don't know how you'd send a POST request in Java, but I'm sure there's a way.
The login.aspx file labels the two fields 'email' and 'password'. Is that all I need to pass in?
Cheers
Blakey
The guy in the green hat.
- jfrazierjr
- Deity
- Posts: 5176
- Joined: Tue Sep 11, 2007 7:31 pm
Re: DDi Compendium API and Java Classes?
You should pass EVERY form element on the page. Example:Blakey wrote:I can easily figure out how to do a POST in java. The question is what do I need to post?Thanlis wrote:I have some perl code that scrapes the Compendium. The sign in process uses POST rather than GET, so you can't pass the parameters via the URL. I don't know how you'd send a POST request in Java, but I'm sure there's a way.
The login.aspx file labels the two fields 'email' and 'password'. Is that all I need to pass in?
Cheers
Blakey
Code: Select all
<input type="submit" name="InsiderSignin" value="Sign In" id="InsiderSignin" />
The BEST way to do this is to use Java to do a GET on the login page itself. Then in the response document, interate though the page's form elements and grab each <input> tags name and value. Then, post those name/value pairs to the login.aspx page. When you get the response from this page, pull the cookie values out of the header and store for later use.
I save all my Campaign Files to DropBox. Not only can I access a campaign file from pretty much any OS that will run Maptool(Win,OSX, linux), but each file is versioned, so if something goes crazy wild, I can always roll back to a previous version of the same file.
Get your Dropbox 2GB via my referral link, and as a bonus, I get an extra 250 MB of space. Even if you don't don't use my link, I still enthusiastically recommend Dropbox..
Get your Dropbox 2GB via my referral link, and as a bonus, I get an extra 250 MB of space. Even if you don't don't use my link, I still enthusiastically recommend Dropbox..