Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Submit feedback
    • Contribute to GitLab
  • Sign in
Rex Dri
Rex Dri
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 32
    • Issues 32
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge Requests 0
    • Merge Requests 0
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Environments
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • CI / CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • Rex Dri
  • Rex DriRex Dri
  • Issues
  • #28

Closed
Open
Opened Jan 22, 2019 by Florent Chehab@chehabfl22 of 22 tasks completed22/22 tasks
  • Report abuse
  • New issue
Report abuse New issue

Handling of automatic data update

⚠ Never ever commit any ID or data from the UTC DB ⚠

UTC Data

UTC data is hosted on an Oracle database.

Step 1

  • Create a new django app in the folder interface_UTC (at the root of the repo). (will be done in Java)
  • The app should connect to the UTC DB and convert the 4 views to JSON arrays,
  • This app must advertise one endpoint for each view,
  • The app must advertise other endpoints that accept a login as parameter:
    • to check what are the sharing permissions associated with a student
    • to get all the departures information for a student
    • So that we don't have to fetch all the data when there is a permission change.
  • The app must be documented, consistent (error code) and resilient,
  • All endpoints will be readonly.

Step 2

  • Create a mock of this new API so that we can easily use it in a dev environment.
  • Don't commit any personal data,
  • You can use a local database (of your choice) to hold the dev data,
  • All the mock data should be of the form of (commited) SQL scripts,
  • You should update the docker-compose file with the new service(s).

Step 3

  • Create the python scripts to handle the transfer of the data from the API created above into Django models.
  • Those scripts should be runnable on demand.
  • Put them in a sub-application of the backend called data_exchange

Currency exchange rate

  • Handle the connection to Fixer API to retrieve the exchange rates (directly from data_exchange)
  • Handle the update of the data in the model
  • Make it resilient.

Both

  • Investigate the use of Cron (inside the backend docker container) to have an automated daily data update. https://uwsgi-docs.readthedocs.io/en/latest/PythonDecorators.html

Finally

  • Create a backend model / and frontend page, to show when where the latest update / the number of errors; so that anyone can see if the system is healthy at any point in time.

  • Also run clearsessions and delete user in the cron 🎉

Edited Jun 29, 2019 by Florent Chehab
To upload designs, you'll need to enable LFS. More information
Assignee
Assign to
PR next
Milestone
PR next
Assign milestone
Time tracking
None
Due date
None
3
Labels
Feature backend ✈✈✈ v1
Assign labels
  • View project labels
Reference: rex-dri/rex-dri#28