Commit 6cad83dd authored by Alexandre Lanceart's avatar Alexandre Lanceart Committed by Florent Chehab

feat(external data): loading currencies

* Added an app in the backend dedicated to the link with external data
* Create a command to update the data
* Support updating data from fixer
* Updated doc accordingly

Almost done #28
parent 6f07c0ea
Pipeline #40900 passed with stages
in 3 minutes and 33 seconds
......@@ -63,6 +63,7 @@ INSTALLED_APPS = [
"rest_framework.authtoken",
"backend_app",
"base_app",
"external_data",
"webpack_loader",
"django_filters",
]
......
from django.contrib import admin
from external_data.models import ExternalDataUpdateInfo
admin.site.register(ExternalDataUpdateInfo)
from django.apps import AppConfig
class ExternalDataConfig(AppConfig):
name = "external_data"
import logging
import os
import requests
from django.core.management.base import BaseCommand
from backend_app.models.currency import Currency
from external_data.models import ExternalDataUpdateInfo
FIXER_API_TOKEN = os.environ["FIXER_API_TOKEN"].strip()
logger = logging.getLogger("django")
class Command(BaseCommand):
help = "Command to handle updating remote data"
def add_arguments(self, parser):
subparsers = parser.add_subparsers(
title="subcommands", dest="subcommand", required=False
)
subparsers.add_parser("all", help="(default) Update all external data")
subparsers.add_parser("currencies", help="Update currencies from fixer")
def handle(self, *args, **options):
if "subcommand" in options.keys():
subcommand = options["subcommand"]
if subcommand == "all" or subcommand is None:
self.update_all()
elif options["subcommand"] == "currencies":
self.update_currencies()
else:
self.update_all()
@staticmethod
def update_all():
logger.info("Updating all external data")
Command.update_currencies()
@staticmethod
def update_currencies():
logger.info("Updating currencies")
response = requests.get(
"http://data.fixer.io/api/latest?access_key={}".format(FIXER_API_TOKEN)
)
data = response.json()
if data["success"]:
for (code, rate) in response.json()["rates"].items():
Currency.objects.update_or_create(
code=code, defaults={"one_EUR_in_this_currency": rate}
)
ExternalDataUpdateInfo.objects.create(source="fixer")
logger.info("Currency update was successful")
else:
logger.error(
"Updating currency information from fixer failed, see response from the API below."
)
logger.error(response.json())
# Generated by Django 2.1.7 on 2019-06-01 09:22
from django.db import migrations, models
class Migration(migrations.Migration):
initial = True
dependencies = []
operations = [
migrations.CreateModel(
name="ExternalDataUpdateInfo",
fields=[
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
("timestamp", models.DateTimeField(auto_now=True)),
("source", models.CharField(max_length=50)),
],
)
]
from django.db import models
class ExternalDataUpdateInfo(models.Model):
timestamp = models.DateTimeField(auto_now=True)
source = models.CharField(
max_length=50
) # Source identifier of the data that was updated
......@@ -29,7 +29,7 @@ services:
networks: [backend-nginx, backend-db]
environment:
WAIT_HOSTS: database:5432 # For the 'wait' script, so that we are sure the db is up and running
env_file: [./server/envs/db.env, ./server/envs/django.env]
env_file: [./server/envs/db.env, ./server/envs/django.env, ./server/envs/external_data.env]
# Run the django developpement server on image startup.
command: /bin/sh -c "/wait && cd backend && ./entry.sh && ./manage.py runserver 0.0.0.0:8000 --nostatic"
# Required that the `database` and `frontend` service are up and running.
......
External data
=============
`REX-DRI` interacts with several external services to provide consistent and up-to-date data to the application.
All the fetching of remote data is handled in a dedicated backend app: `external_data`. A command line interface has also been developed. Once connected to the backend app container, run:
```bash
./manage.py update_external_data --help
```
```txt
usage: manage.py update_external_data [-h] [--version] [-v {0,1,2,3}]
[--settings SETTINGS]
[--pythonpath PYTHONPATH] [--traceback]
[--no-color]
{all,currencies} ...
Command to handle updating remote data
[...]
subcommands:
{all,currencies}
all (default) Update all external data
currencies Update currencies from fixer
```
## Currencies
Currencies exchange rates are taken from [`fixer`](https://fixer.io/). You must provide a consistent API key in `server/envs/external_data.env` to be able to use the service properly.
To trigger un update of currencies, run `./manage.py update_external_data currencies`.
## UTC
TODO
## Auto update
TODO
......@@ -19,6 +19,7 @@
* [Data validation](Application/Backend/data_validation.md)
* [Tags](Application/Backend/tags.md)
* [API](Application/Backend/API.md)
* [External data](Application/Backend/external_data.md)
* [Tests](Application/Backend/tests.md)
* [Optimization](Application/Backend/optimization.md)
......
......@@ -21,7 +21,7 @@ services:
- ./uwsgi/:/usr/src/uwsgi/:ro
- django_logs:/var/log/django
- frontend_logs:/var/log/frontend
env_file: [./envs/db.env, ./envs/django.env]
env_file: [./envs/db.env, ./envs/django.env, ./envs/external_data.env]
environment:
WAIT_HOSTS: database:5432 # For the 'wait' script, so that we are sure the db is up and running
ENV: PROD # make sure to be in prod env
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment