GSoC 2016 Project - Schema update

The concern regarding the updation of data, if the user run the scripts several times has been resolved . Initially I had thoughts of using workshop_id and email_id to identify the user. But as it turned out, we had to capture the data even if a user doesn’t provide their email_id and workshop_id. So as proposed by Piotr Banaszkiewicz, we can use a hash which will be returned by API to the user. This hash can then be saved on the disk.

We have to keep a track of the user’s progress during installation of packages. So everytime a user runs the script, all the data is logged in the tables. But this would require an attempt number to be associated with each request, so that, each attempt can be distinguished from the other. I have added a table Attempts, the primary key of which will be used as foreign key in both tables, and would allow us to distinguish between attempts.

Therefore, on each request a user makes, following things are done:

  • Script checks whether it already has a unique key in the file ‘key.txt’. If yes, it checks whether the date of the key and current date are same. As this key might have been generated for some other workshop held on some other day.
  • If both checks are passed, then it sends this unique key along with the data. Else, unique key is set to None.
  • Server checks whether it already has a ‘user’ record for this unique id. If yes, same user id is used for insertion of data in the tables.
  • Attempts table contains a primary key id which is also the foreign key in ‘successfull_installs’ and ‘failed_installs’ tables. Using this key, attempts/progress of the user can be tracked down.

A unique id is associated with each user and their several attempts. This helps in recognizing the existing user, and thus associating the same user id with failed and succesfull installs. I chose to use UUID over hash, as it is much easier to handle. Refer this link. And it also turns out to be better than hashing technique like SHA1 in some aspects. Refer this link

Summing up, following things have been done till date :


Develop models of database using SQLAlchemy.
Migration of database using Flask-Migrate.
API endpoint “/installation_data/”.
Database connection with API.
Modification of installation script to send data to server (POST request).
Saving key to local disk for unique identification.

TODO :


Sending the error description and causes along with failure and success list.
Collecting User’s system information for sending to server.
Writing Unit tests.
Analysis of results.
Writing the front end to visualize the data.
Testing and Bug fixes.
Testing the scripts on several operating systems.

Project Code: