There are 3 components in ECM:
- A module for retrieving and analyzing data from EVE API servers
- A module for persistence in database
- A module for formatting and presentation of these data
Fetching & Analysis¶
Rather than develop our own module that makes queries, retrieves and decodes the response, we reused an existing component developed by a player: eveapi.
eveapi fetches XML streams from CCP servers and converts them to objects that we can handle. Those objects are handed to ECM analysis system. It will perform multiple operations on the raw data before sending them to the persistence system:
- Filtering extraneous data.
- Pre calculate some indicators such as the level of accreditation of each member (in order not to have to do it on demand).
- Determination of changes in the data analyzed compared to those already in the database.
- Instantiating objects representing the analyzed data. These objects will be interpreted by the persistence system.
Here is a diagram depicting the detailed operation of the module of "Data fetching and analysis":
Already embedded in Django.
To reduce the load on the server, a part of the presentation workload will be performed by the client (web browser). The performance gain is especially noticeable when displaying tables and trees containing a large amount of information.
Once the client has received enough data, it will be able to interact with the server through AJAX requests. The server will then send the data in JSON format to reduce network traffic, the client will perform the presentation job by manipulating the HTML page being displayed.
According to the URL invoked by the client, the server will return either a complete HTML page that will be generated through a "template engine" or raw data in JSON format that will be encoded from the objects from the persistence system.
This method will allow the display of tables and trees containing a large number of elements while limiting the workload of the server.