feat(documentation): more documentation on app specific things

* Documentation of backend specific things
* Docuentation of the use of redux in the frontend
* Cleaned documentation elsewhere

Related to #74

documentation/Application/Backend/config_files.md

@@ -3,35 +3,60 @@ Words relative to *config files*
 
 <!-- This file is referenced in the config_files, if you modify its location, please update those files too. -->
 
+There exist different kinds of *config files* throughout the app and all of them are absolutely necessary, so it's good to understand what is their purpose.
+
+
 ## Django config files
 
-!> TODO
+Django's main config file in `backend/base_app/settings.py`. This file is a standard Django one and defines many important aspects of the app such as:
+
+- Which packages should be bundled with the app,
+- What Middleware to use,
+- Custom behaviors depending on if we are in testing, developing or production environment,
+- Config to serve the frontend files from the backend,
+- App specific settings,
+- Database settings,
+- Etc.
 
 
 ## Custom config files
 
-To centralize the definition of certain certain Models and (all) ViewSets attributes, we centralized a lot of information in few configuration files. Some of those files are also read at compile time when the frontend is built to automatically generate the required `Redux` actions and reducers.
+To centralize the definition of certain certain Models and (all) viewsets attributes, we gathered a lot of information in few configuration files.
+
+-----
+
+?> For your information, some of those files are also read at *frontend-*compile time to automatically generate the required `Redux` actions and reducers.
+
+!> :warning: Models are imported in Django as soon as (and only if) a corresponding viewset is registered in the config files. :warning:
 
-!> TODO more on actions and reducers for the frontend.
+*A viewset is (most often) associated with a serializer class and a serializer class is associated with a model, so we can transparently go up the chain.*
 
-Below is the documentation about those files.
+?> If for some reason you need to bypass this behavior, then you will need to follow the standard way of using Django and register the models in `backend/backend_app/admin.py` and the viewsets in `backend/backend_app/urls.py`.
 
-*NB: all those files use the `yaml` syntax.*
+-----
+
+Below is the documentation about those custom config files that are located in the folder `backend/backend_app/config`, along we some python scripts to help handle them.
+
+*NB: all those files use the awesome `yaml` syntax.*
 
 
 ### defaults.yaml
 
-Contains default values, nothing special to say here.
+Contains default values use across the app, nothing special to say here.
 
 
 ### viewsets_config.yml
 
-This file contains the configuration of the (all) the app viewsets. It has a key/value structure with key the ViewSet name.
+This file contains the configuration of the (all) the app viewsets. It has a key/value (dict) structure with key the viewset name.
 
-Each viewset has the following required required attributes:
+Each viewset entry is defined with the following required required attributes:
 
 * `api_end_pont`: it is the main part of the url for making request to the api. This string will also be used for naming variables in JS, so keep it simple and consistent please. :smile:
-* `import_location`: in what file path (dot like notation) is the viewset available ? You should only specify the part that comes after `backend_app.models.`. *NB: the associated model should be available from the same path.*
+* `import_location`: in what file path (dot like notation - just like any python import) is the viewset available ? You should only specify the part that comes after `backend_app.models.`. *NB: the associated model should be available from the same path.*
+
+!> **If you change an `api_end_pont` you will most likely need to update some JS files (see [here](Application/Frontend/redux.md)).**
+
+TODO direct link to section
 
 
 There can also be optional parameters:
@@ -47,18 +72,16 @@ If no value is taken, then the default one is added in the `load_viewsets_config
 
 ### models_config.yml
 
-This file contains the configuration of the app models. It has a key/value structure with key the model name.
+This file contains the configuration of the app models. It has a key/ (dict) structure with key the model name.
 
 For now, there is only one attribute which can be set: `moderation_level`. This attribute tells how moderation should be applied to each model.
 
-Moderation levels are defined as follow :
-
-* `0` : moderation will never be applied,
-* `1` : moderation will be on if the global settings for moderation is turned on,
-* `2` : (default for security reasons) moderation will always be on no matter what.
+Moderation levels are defined as follow:
 
-*NB: staff members, dri members and moderators won't be subject to moderation.*
+* `0`: moderation will never be applied,
+* `1`: moderation will be on if the global settings for moderation is turned on,
+* `2`: (default for security reasons) moderation will always be on no matter what.
 
-*NB: moderation can be moreover enforced at the object level. But that's for another documentation part.*
+*NB: staff members, dri members and moderators won't be subject to (model-level) moderation.*
 
-!> TODO doc about this.
+*NB: moderation can be moreover enforced at the object level. But that's for [another documentation part](Application/Backend/moderation_and_versioning.md).*

documentation/Application/Backend/models_serializers_viewsets.md

@@ -4,7 +4,7 @@ Models, Serializers and ViewSets
 ## Basics
 
 Our backend app is based on Django. If you are familiar with Django then you will understand what is a Model.
-The general idea is that Django is used to handle the data and nothing else. Lets say that we are only using the "M" of "MVC" of a standard Django architecture.
+For this project, the general idea is that Django is used to handle the data and nothing else. Lets say that we are only using the "M" of "MVC" of a standard Django architecture.
 
 To be able to transfer data to the frontend, we use the famous Django extension [django-rest-framework](https://www.django-rest-framework.org/).
 
@@ -13,9 +13,15 @@ To be able to transfer data to the frontend, we use the famous Django extension
 
 To put it into a nutshell, we have:
 
-- Models that handle all data and the exchanges with the database,
-- Serializers that handle the conversion from models' data to JSON and from JSON to models' data,
-- ViewSets that handle communication with the frontend.
+- **Models** that handle all data and the exchanges with the database,
+- **Serializers** that handle the conversion from models' data to JSON and from JSON to models' data,
+- **ViewSets** that handle communication with the frontend.
+
+
+!> :warning: **Before moving to the details of the models/serializers/viewsets you must now that a lot is automated in the REX-DRI backend. Especially regarding *importing* those classes in Django admin and registering the viewsets with their configuration. All is centralized in *config files* (make sure to read the [doc about them](Application/Backend/config_files.md): if you add a Model/Viewset you WILL need to *register* them in those config files for them to be imported at runtime.** :warning:
+
+*NB: as it is explained in the previously cited [documentation page](Application/Backend/config_files.md), models are registered "through" the associated viewset.*
+
 
 ## In details
 
@@ -23,18 +29,19 @@ Since the process of creating the API was a bit repetitive, we have used a lot o
 
 Below are the things to know about what has been implemented.
 
-
 ### Models
 
-There exist different *abstract* models the "real" models can inherit from and have different behaviors.
+#### App specific
+
+There exist different *abstract* models from which the *concrete* models can inherit from. Those *abstract* models have different behaviors and properties built in directly.
 
 - `MyModel`: contains attribute that should be shared by all models in the app such as for storing who updating something, etc.
 - `MyModelVersionned`: inherits from `MyModel` and make it possible to version the data contain in the models that inherit from it.
 - `BasicModule`: inherits from `MyModelVersionned` and provides a set of attributes such a title, a comment, a custom useful links field and an importance level.
 
-Below will be outdated soon.
+?> :warning: what is said below will soon be unnecessary: https://gitlab.utc.fr/rex-dri/rex-dri/issues/85
 
-!> Models that inherit from `MyModelVersionned` (and therefore `BasicModule`) must have the following calls method:
+Models that inherit from `MyModelVersionned` (and therefore `BasicModule`) must have the following class method that returns the associated serializer:
 
 ```python
 class MyAwesomeClass(MyModelVersionned):
@@ -45,3 +52,152 @@ class MyAwesomeClass(MyModelVersionned):
 ```
 
 This method is used in the `VersionSerializer` class to convert version data back to a model instance.
+
+#### General idea
+
+So to define a model for the app, you should (in most cases) use one of the three base classes defined above. What remains is to define a model just like you would do for any Django Model. Here is an example (you have tons of them in app already):
+
+```python
+from django.db import models
+from backend_app.models.abstract.my_model import MyModel
+
+class Country(MyModel):
+    name = models.CharField(max_length=200)
+    iso_alpha2_code = models.CharField(primary_key=True, max_length=2)
+    iso_alpha3_code = models.CharField(
+        unique=True, max_length=3, default="", blank=True
+    )
+    region_name = models.CharField(max_length=200)
+    region_un_code = models.CharField(max_length=3)
+    sub_region_name = models.CharField(max_length=200, default="", blank=True)
+    sub_region_un_code = models.CharField(max_length=3, default="", blank=True)
+    intermediate_region_name = models.CharField(max_length=200, default="", blank=True)
+    intermediate_region_un_code = models.CharField(max_length=3, default="", blank=True)
+```
+
+### Serializers
+
+#### App specific
+
+To go along the previously mentioned *abstract* models, we have matching serializers that handle all the logic behind rendering the models' field to JSON, validating the data before creating/updating a model instance and handling the specific behaviors (such as moderation — that comes with all models that inherit from `MyModel` — and versioning — that comes with all models that inherit from `MyModelVersionned`).
+
+:warning: **95% of the *magic* behind those not that simple behaviors are handled in a subtle, sometimes *hacky*, manner.**
+
+:information_desk_person: All those specific behaviors should have been correctly unit-tested at this point.
+
+?> You can find more information about moderation and versioning [here](Application/Backend/moderation_and_versioning.md).
+
+
+#### General idea
+
+To create a serializer for your model, you need to create a class that inherits from the matching parent class of your model. For instance the `Country` model inherits from the `MyModel` *abstract* class, so the `CountrySerializer` inherits from `MyModelSerializer`.
+
+Then you must define a `Meta` class inside the serializer that countains information about what model is the serializer for and what fields (attribute) should be taken into account. For instance for the country model we have the following serializer:
+
+```python
+class CountrySerializer(MyModelSerializer):
+    class Meta:
+        model = Country
+        fields = "__all__"
+```
+
+So the serialization of all fields will be handled automatically by `django-rest-framework`.
+
+You may also decide to exclude some fields:
+
+```python
+class UniversitySerializer(MyModelSerializer):
+    class Meta:
+        model = University
+        exclude = ("utc_id",)
+```
+
+And even include custom serializer to make some fields read only, or add data to send to the client:
+
+```python
+class MyModelSerializer(MySerializerWithJSON):
+    # ...
+    # Extract of the definition of the class
+
+    # This field is "smartly" set by the serializer on save
+    # and should be considered as readonly.
+    updated_by = serializers.CharField(read_only=True)
+
+    # For easier handling on the client side, we force an id field
+    # this is useful when a model has a dedicated primary key
+    id = serializers.SerializerMethodField()
+
+    def get_id(self, obj: MyModel):
+        """
+        Serializer for the id field.
+        """
+        return obj.pk
+
+    class Meta:
+        model = MyModel
+    # ...
+```
+
+As shown above, when using a `custom_attr = serializers.SerializerMethodField()`, you need to define a method in the class that is named `get_custom_attr` and returns whatever you want (as soon as it can be converted to JSON).
+
+
+?> Django-rest-framework is full a functionalities, and only a subset of them have been presentend here. So feel free to have a look at [the package documentation](https://www.django-rest-framework.org/).
+
+
+### Viewsets
+
+#### App specific
+
+The custom work performed on the generic viewsets is fairly straightforward, you can have a look at it. Anyway, just like with the serializer, you should use the corresponding viewset when inheriting: `MyModelViewset` or `MyModelVersionnedViewset` or `BasicModuleViewSet`.
+
+The general idea is that a good chunk of the parametrization of the viewsets have been moved away from the class definition to the [config files](Application/Backend/config_files.md).
+
+
+#### General idea
+
+Most often a wiewset class will look like this, with a simple reference to the associated serializer class:
+
+```python
+class MyModelVersionnedViewSet(MyModelViewSet):
+    """
+    Viewset for the versionned models
+    """
+    serializer_class = MyModelVersionnedSerializer
+```
+
+Sometimes you will also need to filter the data against some attribute found in the client request:
+
+```python
+class CountryDriViewSet(BasicModuleViewSet):
+    queryset = CountryDri.objects.all()  # pylint: disable=E1101
+    serializer_class = CountryDriSerializer
+
+    def get_queryset(self):
+        country_id = self.kwargs["country_id"]
+        return super().get_queryset().filter(countries__pk=country_id).distinct()
+```
+
+Or also prefetch some related attributes for optimization purposes:
+
+```python
+# Extract of the class definition
+class MyModelViewSet(viewsets.ModelViewSet):
+    def get_queryset(self):
+        """
+        Extended default rest framework behavior
+        to prefetch some table and enhance performances
+        """
+        return self.queryset.prefetch_related("moderated_by", "updated_by")
+```
+
+?> If you are familiar with `REST` api, you will surely be familiar with filtering elements directly in the request with a syntaxe such as `url/?username=bob&isOld=true`. This syntax is not yet supported, since we didn't need it up until now.
+
+
+## Keeping things consistent
+
+!> :warning: To keep things consistent the general idea is that if a model is called `Blabla` than it's serializer class will be named `BlablaSerializer` and the associated viewset class will be named `BlablaViewSet`.
+
+:information_desk_person: Sometimes it will be useful to have different viewsets for the same serializer, for instance if you want an api endpoint that returns the data filtered in a certain way.
+
+!> You should avoid having multiple serializers for the same model since it may brake things related to deserializing previous versions of models.
+

documentation/Application/Backend/moderation_and_versioning.md

+Moderation & Versionning in the app
+=================================
+
+## Moderation
+
+### Application level
+
+!> TODO
+
+### Model level
+
+!> TODO
+
+### Object/instance level
+
+!> TODO
+
+
+## Versioning
+
+### General idea
+
+!> TODO

documentation/Application/Backend/optimization.md

+Few words on optimization
+===============
+
+TODO optimization of the request.

documentation/Application/Frontend/react.md

+Quick intro to React
+===============
+
+?> :information_desk_person: `React` is a JavaScript library to build user interface and it was introduced by Facebook.
+
+
+## General idea
+
+When doing web development, you should be familiar with the *DOM* ([Document Object Model](https://fr.wikipedia.org/wiki/Document_Object_Model)) which is basically a tree translation of the HTML file you send to the browser for rendering.
+
+React is built on a *virtual DOM* (you can explore it with the [React Developer Tools](https://chrome.google.com/webstore/detail/react-developer-tools/fmkadmapgofadopljbjfkapdkoienihi) extension for chromium based browsers). The elements of this *virtual DOM* are no longer HTML tags; they are real *living* objects called **components** that can be modified et that can perform multiple actions. Every time an update to is performed, the DOM in your browser is smartly updated by react based on its own internal *virtual DOM*.
+
+Components being elements of tree like structure, they must have a parent (or be the root node) and they can have as many *children* components as they want.
+
+
+## TODO more
+
+

documentation/Application/Frontend/redux.md

+Use of redux
+======
+
+
+?> `Redux` is a predictable state container for JavaScript apps.
+
+
+Why redux?
+
+
+As you may now, react components have a strict living period and as soon as they are *unmounted* from the DOM the data they contained in their state is completely deleted. The redux object (called a *store*) on the other hand will live as long as your app is open in the browser so it’s a nice place to store information in.
+
+
+*NB: it lives as long as your browser is open because it usually is the root component of your react app, as you can see in `frontend/src/index.js`*.
+
+
+Redux solves another issue which is that sometimes different components in different locations of your *virtual DOM* need to access similar data. So redux is very useful to store data from a backend API.
+
+
+
+## How it works
+
+
+To understand how redux works, you need to understand 2 principles: *actions* and *reducers*.
+
+
+!> *Actions* are objects/functions your app *dispatches* and that contains a `type` (to identify the action) and some optional contextual data.
+
+
+!> *Reducers* are objects/functions that listen to the *actions* being dispatched and update the redux `store`'s **state** depending on the action and the contextual data they provide.
+
+
+
+### Example of an action
+
+
+*Taken from `frontend/src/redux/actions/map.js`.*
+
+
+```js
+export function saveMainMapStatus(newStatus) {
+  return {
+    type: SAVE_MAIN_MAP_STATUS,
+    newStatus
+  };
+```
+
+
+This action is of type `SAVE_MAIN_MAP_STATUS` (defined in `frontend/src/redux/actions/action-types.js`) and is called with a *new status* (a javascript Object) as contextual data.
+This action is used to save the state of the map (what is the zoom level, what are the coordinates of the center and what layer is selected) in our application.
+
+
+
+### Example of a reducer
+
+
+To interpret this action when it is *dispatched* we have the following *reducer*:
+
+
+```js
+export function saveMainMapStatus(state = { center: [0, 2], zoom: 2, selectedLayer: "OpenStreetMap France" }, action) {
+  switch (action.type) {
+    case SAVE_MAIN_MAP_STATUS:
+      return {
+        center: action.newStatus.center,
+        zoom: action.newStatus.zoom,
+        selectedLayer: action.newStatus.selectedLayer
+      };
+
+
+    default:
+      return state;
+  }
+}
+```
+
+
+So a *reducer* is given the previous state of the redux *store* (or here a portion of the store, this will be explained later) and an *action*. If the action "concerns" the reducer then it returns a new state based on the action contextual data; otherwise, the state is returned unchanged.
+
+
+The idea is that we have tons of reducers in our app, and we compose them to have a better organization. So each of those reducers will be called for all dispatched actions, hence the simple `switch` statement to check if this reducer is concerned by the action.
+
+
+
+
+### General shape of our app's redux store
+
+
+In *REX-DRI*, all the reducers are composed in such a way that the final store looks like this:
+
+
+```js
+{
+    app: {
+        // data that concerns the app and only the app
+    },
+    api: {
+        // data that concerns all the endPoint provided by the backend API
+    }
+}
+```
+
+
+The data in `app` is build with reducers handwritten, just like the one above. The data in `api` is based on reducers generated automatically at runtime based on the app [config files](Application/Backend/config_files.md).
+
+
+
+## Accessing the data in the redux store
+
+
+### Direct access
+
+
+A direct access to the redux store is possible and very simple: you need to call the `getState` method on the store defined in `frontend/src/redux/store.js`. `getState` returns the full current state of your store.
+
+
+```js
+import store from "../store";
+const myData = store.getState().app.//....
+```
+
+
+?> :information_desk_person: some helpers functions have been built in `frontend/src/redux/api/utils.js` to handle this access more easily in some circumstances.
+
+
+
+Such a direct access is, however, not always interesting...
+
+
+
+### Subscribing to the store
+
+
+Redux make it possible to make a component subscribe to the store or a portion of it, and when it's modified, then your component will be automatically updated. When we say "subscribe" we mean that:
+
+
+- Redux will add the `props` you tell him to your components,
+- Those props map to a portion of the redux store (/state),
+- Every time that portion of the store is updated, the props on your components will be updated,
+- And finally, given how react is built and because your props have been updated, your component will be updated. :confetti_ball:
+
+
+Here is an implementation example:
+
+
+```js
+import React, { Component } from "react";
+import { connect } from "react-redux";
+
+class UnivMap extends Component {
+    render(){
+        return (<>{this.props.map}</>)
+    }
+}
+
+const mapStateToProps = (state) => {
+  return {
+    map: state.app.mainMap
+  };
+};
+
+export default connect(mapStateToProps)(UnivMap);
+```
+
+
+When exporting your component, you wrap it with the `connect` function provided by redux. This function takes as argument another `mapStateToProps` function.
+
+
+`mapStateToProps` is a function that takes as argument the state (of the redux store) and returns an object with key(s) the `props` that will be added to your component and value(s) the portion(s) of the store you are interested in (and that will be accessible from the `props`).
+
+
+And then everything is automated! :confetti_ball:
+
+
+
+## Putting data in the store
+
+
+Well, it's nice to be able to read data from the redux store, but it would be even cooler to be able to put data in the store.
+
+
+There comes in the actions (and indirectly the reducers) you have defined earlier :smile:.
+
+
+The idea is that you want your component to be able to *dispatch* actions (that will be handled by the reducers and that will update the state of the redux store).
+This is done once again with redux `connect` function:
+
+
+
+```js
+import React, { Component } from "react";
+import { connect } from "react-redux";
+import { saveMainMapStatus } from "../../redux/actions/map";
+
+
+
+class UnivMap extends Component {
+  componentWillUnmount() {
+    // ...
+    this.props.saveMainMap({
+      zoom: leafletInstance.getZoom(),
+      center,
+      selectedLayer
+    });
+    // ...
+  }
+
+  render() {
+    return (<> />)
+  }
+}
+
+const mapDispatchToProps = (dispatch) => {
+  return {
+    saveMainMap: (mapState) => dispatch(saveMainMapStatus(mapState)),
+  };
+};
+
+export default connect(() => {}, mapDispatchToProps)(UnivMap);
+```
+
+
+So this time, we have the `mapDispatchToProps` function to pass to the `connect` function from redux.
+
+
+`mapDispatchToProps` takes as argument the *dispatch* capability provided by redux and returns an object. Once again, the key(s) of this object will be mapped to your component's `props`. Those new `props` will be functions that take optional arguments and then *dispatch* a specific action.
+
+
+That's all! :confetti_ball:
+
+
+?> :information_desk_person: as you can see the `connect` function from redux takes as first argument a `mapStateToProps` function and as second argument a `mapDispatchToProps` function. Providing the second one is optional. To provide only the second one, you can put `() => {}` (useless function) as first argument.
+
+
+
+
+## Redux and the backend API
+
+
+!> :warning: This section if probably the most important if you are already familiar with redux. Because we have tons of backend API endpoints, we have implemented a generic way to use them with redux. :warning:
+
+
+
+### Dynamic actions and reducers
+
+
+
+
+As you can see above, writing actions and reducers is very verbose and redundant, especially if you have a lot of them that only differ from a name or endpoint.
+
+
+So... actions related to the API are generated at runtime based on the [configuration]((Application/Backend/config_files.md)) of all available endpoints. The same goes for the associated *reducers* that are then combined under `state.api` in the redux store. :confetti_ball: