Rex Dri issueshttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues2020-06-14T19:16:22Zhttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/192Stats explorer2020-06-14T19:16:22ZFlorent ChehabStats explorerFollowup to #184
We are close to having the following data collected everyday on REX-DRI:
1. How many users connected to the plateforme, with a 24h granularity.
2. Information about exchange related contributions with a 24h granulari...Followup to #184
We are close to having the following data collected everyday on REX-DRI:
1. How many users connected to the plateforme, with a 24h granularity.
2. Information about exchange related contributions with a 24h granularity.
## Example data
Here is a preview how what the two tables might contain:
1. ex:
| *id* | date | nb_connections |
| ---- | ------------------------- | -------------- |
| 1 | 2020-05-01T00:00:00:0001Z | 5 |
| 2 | 2020-05-02T00:00:00:0001Z | 10 |
| 3 | 2020-05-03T00:00:00:0001Z | 3 |
2. ex:
| *id* | date | major | minor | exhange_semester | university | nb_contributions |
| ---- | ------------------------- | ----- | ----- | ---------------- | ---------- | ---------------- |
| 1 | 2020-05-01T00:00:00:0001Z | IM | CMI | P2020 | EPFL | 2 |
| 2 | 2020-05-01T00:00:00:0001Z | IM | MAT | A2020 | EPFL | 1 |
| 3 | 2020-05-03T00:00:00:0001Z | GI | FDD | A2020 | Tokyo | 1 |
*N.B. 'university' will actually be a foreign key to the univ model in the database.*
## Objectives
Those tables enable us to answer the following questions:
1. Questions:
* How many user connected to the site in the last 30 days?
* How many user connected to the site between date1 and date2?
* Can we plot the number of user that connected to site between date1 and date2?
* Etc.
2. Questions (here there are a lot of possibilities -- plus either count or plot):
* How many exchange related contributions between date1 and date2?
* What is the distribution of contributions between majors?
* ... between minors inside a major?
* etc. etc. etc.
## Technical feasibility
### Request
In the frontend, we want to:
* Be able to plot data or display it as a table,
* Be able to export tables as CSV,
* Be able to filter data,
* Be able to "groupby" data,
* Be able to combine the last two settings in whatever configuration,
* Be able to share a reproductible page (filter, groupby, etc.) only with a URL.
### Constraints
* There are a lot of filtering/groupby possibilities, our tool has to be generic enough to cover every possibility.
* The solution should be highly maintainable.
* We must have a strict control about what query is performed in the database.
* We can assume that the end user has some data wrangling knowledge.
* It should be implementable under ~2 weeks.
### Conclusion
How to do it?PR-P20Estelle VeisemburgerEstelle Veisemburgerhttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/187Dramatically ease the process of adding missing destinations2020-05-31T12:39:13ZFlorent ChehabDramatically ease the process of adding missing destinations:warning: to be tackled after #137 !
In REX-DRI, there is a difference between a UTC university partner and a University (for REX-DRI).
This is because:
* There are duplicates in the UTC database,
* We want to add Metadata to the Unive...:warning: to be tackled after #137 !
In REX-DRI, there is a difference between a UTC university partner and a University (for REX-DRI).
This is because:
* There are duplicates in the UTC database,
* We want to add Metadata to the University (the location of their campus, etc.)
As a result we have the [`Partner` model](https://gitlab.utc.fr/rex-dri/rex-dri/-/blob/dev/backend/backend_app/models/partner.py) which is automatically filled.
And then, we need to either:
* Be able to link this partner to an existing university at REX-DRI,
* Create a new University at REX-DRI.
After #137 is tackled, it should be straightforward to do it. Document the process (using the django admin).
You might also want to update this view https://rex.dri.utc.fr/app/about/unlinked-partners/ to provide the id in the database to ease the process.PR-P20Maxime EmschwillerMaxime Emschwillerhttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/184Add stats regarding feedback completion2020-06-14T13:25:19ZFlorent ChehabAdd stats regarding feedback completionCompute basics statistics (dayly) regarding what is the percentage of exchange with feedbacks, how many new feedbacks, etc.Compute basics statistics (dayly) regarding what is the percentage of exchange with feedbacks, how many new feedbacks, etc.PR-P20Maxime EmschwillerMaxime Emschwillerhttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/180Update documentation about api interactions frontend / backend2020-04-14T19:46:00ZFlorent ChehabUpdate documentation about api interactions frontend / backendFlorent ChehabFlorent Chehabhttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/177Filter: give more suggestions & merge IM, GSM, GM ?2020-05-24T18:53:21ZFlorent ChehabFilter: give more suggestions & merge IM, GSM, GM ?> Dans la recherche, si je commence à taper « IM » dans le filtre par branche, seules quelques filières apparaissent. Je ne trouve pas « IM-CMI ». Il faut taper CMI pour faire apparaître IM-CMI et GSM-CMI (nécessité de garder les deux ?)> Dans la recherche, si je commence à taper « IM » dans le filtre par branche, seules quelques filières apparaissent. Je ne trouve pas « IM-CMI ». Il faut taper CMI pour faire apparaître IM-CMI et GSM-CMI (nécessité de garder les deux ?)PR-P20Estelle VeisemburgerEstelle Veisemburgerhttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/149Last visited universities2020-06-07T12:16:19ZFlorent ChehabLast visited universitieshttps://gitlab.utc.fr/rex-dri/rex-dri/issues/149
Backend:
1. Create an intermediate model (LastVisitedUniversity) between the user and university models
* This model contains 3 fields : one for user (foreign key) , one for universi...https://gitlab.utc.fr/rex-dri/rex-dri/issues/149
Backend:
1. Create an intermediate model (LastVisitedUniversity) between the user and university models
* This model contains 3 fields : one for user (foreign key) , one for university (foreign key) and one for lastClickOnUniversity (DateTimeField: auto_now = True)
2. Create a viewset to update the previous model when a user looks at a university (get inspired by https://gitlab.utc.fr/rex-dri/rex-dri/-/blob/dev/backend/backend_app/viewsets.py#L316).
Endpoint: /lastVisitedUniversities
req POST endpoint/<pk_univ> => Create LastVisitedUniversity(user=req.user, university=University.get(pk=<pk_univ>), ~~ts=now()~~)
req GET endpoint => return List[LastVisitedUniversity.where(user=req.user).orderby(ts DESC).first(5)] (list[int] === List[<pk_univ>])
SQL:
```sql
SELECT university, max(ts) as ts
FROM LastVisitedUniversity
WHERE
USER = <req.user.pk>
GROUP BY university
ORDER BY ts DESC
LIMIT 5;
```
other => Return 404
Frontend:
1. When visiting university: make POST request (https://gitlab.utc.fr/rex-dri/rex-dri/-/blob/dev/frontend/src/components/recommendation/view/View.jsx#L676 use the useApiCreate() hook)
2. Create component to show last visited univ (and mention REX-DRI private)
PR-P20Imane MisrarImane Misrarhttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/148Filter only on currently available destinations2020-05-17T15:54:44ZFlorent ChehabFilter only on currently available destinations## Objective
* Being able to filter only on open destinations, [on the map and search pages (it's the same component)](https://rex.dri.utc.fr/app/map/)
## Context
### Back
* Every night, the UTC DRI data are copied from the DSI to RE...## Objective
* Being able to filter only on open destinations, [on the map and search pages (it's the same component)](https://rex.dri.utc.fr/app/map/)
## Context
### Back
* Every night, the UTC DRI data are copied from the DSI to REX-DRI [source](https://gitlab.utc.fr/rex-dri/rex-dri/-/blob/dev/backend/cron_tasks.py#L38)
* You should be able to plug yourself in the [`__import_opened_destinations` method](https://gitlab.utc.fr/rex-dri/rex-dri/-/blob/dev/backend/external_data/management/commands/utils.py#L199) and the [`update` method](https://gitlab.utc.fr/rex-dri/rex-dri/-/blob/dev/backend/external_data/management/commands/utils.py#L76), to track which partners are "open" on the UTC ENT.
* You should also know (and maybe use -- it's up to you) that some denormalized data is updated every night on the [`University` model](https://gitlab.utc.fr/rex-dri/rex-dri/-/blob/dev/backend/backend_app/models/university.py#L72). This is the data that is used for filtering.
### Front
* The filter Component is [here][https://gitlab.utc.fr/rex-dri/rex-dri/-/blob/dev/frontend/src/components/filter/Filter.jsx)
* The filtering itself is handled within the [`FilteringService` class](https://gitlab.utc.fr/rex-dri/rex-dri/-/blob/dev/frontend/src/services/FilterService.js)
Have fun :tada:
Bonus point if you throw some [transaction](https://docs.djangoproject.com/fr/2.2/topics/db/transactions/#controlling-transactions-explicitly) somewhere :rolling_eyes: PR-P20Imane MisrarImane Misrarhttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/141Increase uwsgi request header buffer size2019-07-02T20:30:16ZFlorent ChehabIncrease uwsgi request header buffer sizeThe request header is too big for some users and uwsgi crashs.The request header is too big for some users and uwsgi crashs.https://gitlab.utc.fr/rex-dri/rex-dri/-/issues/140Update extra denormalization failing in Cron2019-07-02T20:30:16ZFlorent ChehabUpdate extra denormalization failing in Cronhttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/137Drop the city and campus model to simplify things a bit2020-06-19T14:20:56ZFlorent ChehabDrop the city and campus model to simplify things a bitAs you can see here https://rex-dri.gitlab.utc.fr/rex-dri/documentation/generated/architecture.svg the backend models architecture is a bit complicated.
To simplify things a bit, we want to:
* [ ] Drop the city model
* [ ] Drop the c...As you can see here https://rex-dri.gitlab.utc.fr/rex-dri/documentation/generated/architecture.svg the backend models architecture is a bit complicated.
To simplify things a bit, we want to:
* [ ] Drop the city model
* [ ] Drop the campus model
* [ ] All these information should be moved back to the `University`.
* [ ] There are also big changes to perform to the frontend.
This will be a big help for #187
How to do it?
1. Create a migration to add the missing columns to the University model,
2. Create a script to transfer the data from the city / campus models in the new columns,
-----
3. Update the backend serializers and viewsets,
5. Update the frontend.
-----
4. Delete the old models.PR-P20Gautier DauresGautier Daureshttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/133Update dependencies2019-09-02T17:21:53ZFlorent ChehabUpdate dependenciesAlso:
remove react-swipeable-views, not used anymore in searchAlso:
remove react-swipeable-views, not used anymore in searchhttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/127Recommendation lists: JSON validation and rebuild universities2019-06-16T18:53:44ZFlorent ChehabRecommendation lists: JSON validation and rebuild universitiesFlorent ChehabFlorent Chehabhttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/123Support pagination in DRF and frontend2019-06-30T16:23:30ZFlorent ChehabSupport pagination in DRF and frontendEspecially this endpoint has to be paginated: http://localhost:8000/api/exchangeFeedbacks/Especially this endpoint has to be paginated: http://localhost:8000/api/exchangeFeedbacks/https://gitlab.utc.fr/rex-dri/rex-dri/-/issues/122Preparatory work regarding site charter acceptance2019-06-01T08:55:18ZFlorent ChehabPreparatory work regarding site charter acceptance* Add a bollean field to the user model (defaults to false) to tell if the user has accepted the sites charter and data processing rules,
* Create a Django view that holds a dumb charter and rule. If the user hasn't accepted them, add a ...* Add a bollean field to the user model (defaults to false) to tell if the user has accepted the sites charter and data processing rules,
* Create a Django view that holds a dumb charter and rule. If the user hasn't accepted them, add a button to accept the rules.
* Update the loginRequieredMidlleware to redirect all request to that view until the user accept the rules.
* The view must be accessible even if the user has already accepted the rules.https://gitlab.utc.fr/rex-dri/rex-dri/-/issues/121Add the ability to ban user from the site2019-06-01T08:55:18ZFlorent ChehabAdd the ability to ban user from the site* Update the project's user model and add `is_banned` boolean field
* Add a Django view to roughly tell the user that he was banned (if he was; the view should only be accessible to banned user -- return 404 otherwise)
* Update the login...* Update the project's user model and add `is_banned` boolean field
* Add a Django view to roughly tell the user that he was banned (if he was; the view should only be accessible to banned user -- return 404 otherwise)
* Update the loginRequiredMiddleware (and maybe rename it) to redirect all banned user's requests to the viewhttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/120Add maintenance info2019-06-27T20:03:58ZFlorent ChehabAdd maintenance infohttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/119Check on_delete values in models2019-07-01T18:42:30ZFlorent ChehabCheck on_delete values in modelsBefore going in productionhttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/118Check allowed HTTP method allowed on the API2019-05-25T13:22:57ZFlorent ChehabCheck allowed HTTP method allowed on the APIAllow only GET, POST, PUT and DELETE.Allow only GET, POST, PUT and DELETE.Before going in productionFlorent ChehabFlorent Chehabhttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/116Handling of languages2019-05-25T18:54:03ZFlorent ChehabHandling of languagesScrap /extract csv from: https://fr.m.wikipedia.org/wiki/Liste_des_codes_ISO_639-1
And a or multiple languages foreing key where relavantScrap /extract csv from: https://fr.m.wikipedia.org/wiki/Liste_des_codes_ISO_639-1
And a or multiple languages foreing key where relavantSegolene BrisemeurSegolene Brisemeurhttps://gitlab.utc.fr/rex-dri/rex-dri/-/issues/112Switch to standard JSON schema validation in the backend2019-04-21T17:52:19ZFlorent ChehabSwitch to standard JSON schema validation in the backendhttps://json-schema.org/understanding-json-schema/index.htmlhttps://json-schema.org/understanding-json-schema/index.html