Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now

Roshpit Champions Official Blog


Getting updated Item Schema WITH IMAGES for Dota2 - despite app570 econ API being down for 5 months.

Item Econ
Guide


Last Updated: May 2nd, 2015

In this guide, we will be extracting all the up-to-date image files from the Dota 2 Game Client, and getting detailed item info from the readily available items_game.txt.

We got tired of waiting for the api at: http://api.steampowered.com/IEconItems_570/GetSchema/v0001/?key={yourKey}, which has been returning an empty array since December 18th. This API call was crucial for getting all the image files associated with each item, along with other item details.

Imgur Broken Images everywhere

With this guide, nobody should be blocked from making creative Dota 2 content regardless of the econ API status.

1. Download GFScape

In order to extract the necessary assets, we need some software that can dig through the Dota 2 client's files. I used GFScape. You can google it or download from http://nemesis.thewavelength.net/?p=26

2. Open the Dota2 VPK in GFscape

In order to get through this step, you must have Dota2 game client installed on your computer.

Open GFScape and then hit ctrl+O to browse your computer for the pak01dir.vpk. Your VPK location might slightly differ, but mine was located at
C:\Program Files (x86)\Steam\steamapps\common\dota 2 beta\dota\pak01
dir.vpk

Imgur

3. Locate the econ folder in the Dota VPK

Press ctrl+F to start a find lookup, type in the string 'econ' and hit Find

Imgur

4. Extract the assets

Right click on the 'econ' folder that has a Size of 30 items, and click Extract. You will be prompted to choose where on your computer to extract all of the necessary files.
Imgur

Congrats! We now have all the image files that are unavailable through the API. But how can we use them in conjuction with the data that is available? Read on!

5. Upload images

We need to host the image files online somewhere so that our App or Website can access them. It is important that wherever we upload them, we maintain the file/folder structure because the image paths are referenced in the items_game.txt.

I found the best service for this to be AWS S3. I like this service because it is incredibly fast, free (if you want), and allows you to link over https. It also maintains the folder structure when you upload which is necessary for what we want to accomplish. You will have to make an account and follow the steps to upload. This process is out of the scope of this article, but it's not hard.

6. Acquire the items_game.txt

Despite the standard econ API being down, we can still access the items_game.txt which contains all of the econ data that we need, albeit in a difficult to parse format.

Go to http://api.steampowered.com/IEconItems570/GetSchemaURL/v0001/?key={yourApiKey} to get a link to the most update itemsgame.txt

The result returned is in JSON format, so if you have a browser addon like JSONView you'll see something like this:
Imgur

Click that link or ctrl+c it into your browser to open up the most up todate items_game file.

7. Convert items_game.txt to JSON

The items_game.txt is difficult to parse because it is actually in VDF - Valve Data Format. Luckily some genius by the name of Rob Jackson has created an online VDF to JSON converter.

You can find a link to it here: https://rjackson.me/tools/vdfjson/

You will want to go back to your items_game.txt tab, hit ctrl+A to select all,

Imgur

and then paste it into VDFJson:

Imgur

Be aware, copying and pasting this many lines of text, as well as converting it to JSON, can take some time - so it may feel like your browser has stopped responding when it actually just takes a while. Once it is done, copy the output from VDF Json and paste it into a blank txt file on your computer - and save it as items_game.json (the extension is important).

You now want to host your new json file online so that your application can parse through it at will.

8. Final Steps

Roshpit is programmed in ruby, so below we have posted the ruby code to parse through our items_game.json. Even if you are not using ruby, the following might help you overcome some of the roadblocks we hit.

Imgur

Line 69 - Item Name Encoding
Notice we had to apply encoding options to the item name field. This is because some of the item names in the items_game.txt contain nom UTF-8 characters and can't be stored in a standard database. Lines 62-69 remove all non UTF-8 characters.

Lines 83 & 84

On line 83 we convert all image links to downcase - for some reason the items_game.txt has capital letters in the file paths but the dota 2 client is all lowercase.

On line 84, we reference the images that we uploaded by taking the path found in items_game.txt, prepending our amazon bucket URL and appending .png

Questions

Do we have to keep updating everything every time there are new items added to the Valve stores ?

If you want to get the latest data - yes. The way around this would be through finding a way to automate the steps of this guide.

Since our website only requires the data for tradeable items, we only have to repeat this process once a month because items don't become tradeable until 1 month after they are added to the schema.

If you are developing some sort of library of all items including non-tradeable, you'd want to update with each patch, and finding a way to automate these steps would be best for your service.


We hope you found this article useful. Hopefully nobody will have to struggle over this issue anymore. If you have any questions or comments, please post them on our forum.