Commit 3a615383 authored by Florent Chehab's avatar Florent Chehab
Browse files

feat(standard filtering, tweaks): REST Api and other

* Filtering on client request should now be performed with the standard `?attr=...` syntaxe
* Frontend updated for this new syntaxe
* Backend and frontend documentation updated with new changes
* Updated the location of the the api documentation to `/api-doc`
* Fixed bug preventing api-doc to render
* backend python requirements updated
* Updated dockerfile / docker-compose to make sure we wait for the db

Fixes #97 #80
parent 243f43bf
......@@ -141,6 +141,7 @@ Just like in the standard use of the Django Rest Framework, you should set the f
* `queryset` (or `get_queryset`): the queryset associated with the viewset,
* `serializer_class`: what is the serializer class to use to serialize the queryset.
* `viewset_permission` (defaults to `(IsAuthenticated & (IsStaff | NoDelete))`): what are the permissions associated with each viewset. If something is specified then it will be composed (`&`) with the default one. You can compose permissions with `&` (logical *and*) or `|` (logical *or*). Most permissions are defined in `backend/backend_app/permissions/app_permissions.py`, you can add yours too.
* `filterset_fields` (optional): more information [here](Application/Backend/models_serializers_viewsets?id=filtering)
?> :information_desk_person: If you set `viewset_permission`, it must be a tuple; (in our case it will often have only one attribute).
......@@ -165,19 +166,54 @@ class CountryViewSet(BaseModelViewSet):
end_point_route = "countries"
```
Sometimes you will also need to filter the data against some attribute found in the client's request:
#### Filtering
Sometimes you will also need to filter the data against some attribute found in the client's request.
You should favor the standard approach to do so. If the endpoint is `/api/endpoint` and you want to make sure the model attribute `attr` equals `x` for the object returned then the client should fetch the following route: `/api/endpoint?attr=x` (if multiple attributes need to be filtered then it would look like `/api/endpoint?attr1=x&attr2=y&attr3=z`).
To make it work like this, you must add `filterset_fields` to your viewsets. It should be a tuple of the names of the fields you allow filtering on.
```python
class CountryDriViewSet(ModuleViewSet):
queryset = CountryDri.objects.all() # pylint: disable=E1101
serializer_class = CountryDriSerializer
permission_classes = (IsStaff | IsDri | NoPost,)
end_point_route = "countryDri"
filterset_fields = ("countries",)
```
This filtering is achieved through [`django-filter`](https://django-filter.readthedocs.io/en/master/) package (and the [standard api provided by `django-rest-framework`](https://www.django-rest-framework.org/api-guide/filtering/)), you can further customize the filtering if you'd like to.
------
There exist an other way of filtering directly on the client request, **but you should use it only when there are absolutely no other alternatives or when parameters are absolutely required**. You can see this other way below:
* Endpoint's attributes are set to be captured in `end_point_route`,
* We make use of them in `get_queryset`.
```python
class VersionViewSet(BaseModelViewSet):
# ...
end_point_route = (
r"versions/(?P<content_type_id>[0-9]+)/(?P<object_pk>[0-9A-Za-z]+)"
)
def get_queryset(self):
country_id = self.kwargs["country_id"]
return super().get_queryset().filter(countries__pk=country_id).distinct()
content_type_id = self.kwargs["content_type_id"]
object_pk = self.kwargs["object_pk"]
ct = ContentType.objects.get_for_id(content_type_id)
model = ct.model_class()
obj = model.objects.get(pk=object_pk)
return Version.objects.get_for_object(obj)
```
Or also prefetch some related attributes for optimization purposes:
#### Performance
!> **Viewsets can be huge bottlenecks for performances.**
If the objects the viewset will be serializing have a lot of "foreign" data (from a ForeignKey, ManyToMany field, etc.), you **shall** extend the queryset to make sure to prefetch all the data needed for the serializer:
```python
# Extract of the class definition
......@@ -192,10 +228,7 @@ class EssentialModuleViewSet(BaseModelViewSet):
return self.queryset.prefetch_related("moderated_by", "updated_by")
```
TODO below will be outdated soon
?> If you are familiar with `REST` api, you will surely be familiar with filtering elements directly in the request with a syntaxe such as `url/?username=bob&isOld=true`. This syntax is not yet supported, since we didn't need it up until now.
?> :information_desk_person: This will basically generate an SQL `JOIN` once per queryset. If you don't do this, the serialization of each attribute will result to one `SQL` query, leading to hundreds of them on a simple client request.
## Keeping things consistent
......
......@@ -225,13 +225,19 @@ const mapDispatchToProps = (dispatch) => {
The `getActions` function will give you access to all of the following functions:
- `readAll()`: reads all the data from the endpoint (data will be given as an array of object)
- `readSpecific(id)`: reads the data for an instance (the `id` is the instance's `id`/primary key) (data will be returned as an object)
- `readAll(params={})`: reads all the data from the endpoint (data will be given as an array of object)
- `readSpecific(id, params={})`: reads the data for an instance (the `id` is the instance's `id`/primary key) (data will be returned as an object)
- `create(data)`: creates an object on the endpoint with attributes values given by `data`.
- `update(data)` (`data` must have an `id` field): updates the instance identified by `id`.
- `setInvalidatedAll()`: invalidates "all" the data for the endpoint,
- `setInvalidatedSpecific()`: invalidates "specific" data from the endpoint.
?> `params` is an object that accepts two keys: `queryParams` and `endPointAttr`. In most cases it should be created through the `getQueryParams` and `getEndPointAttr` (and `mapDispatchToProps`; see just before [this section](Application/Frontend/redux?id=words-on-updating-data-on-the-api)) functions of your components.
> * `queryParams` should be an object that maps the fields to the values you want to filter on (`queryParams = {university: 1, country: 2}` will render as `/api/endpoint?university=1&country=2`).
> * `endPointAttr` should be an array of the endpoint attributes to add to the endpoint (`endPointAttr = [10, 11]` will render as `/api/endpoint/10/11/`)
!> Not all actions might be performed on any given endpoint: your request my get rejected by the backend depending on the viewset's `permission_classes`, the object, the user who sent the request, etc.
?> :information_desk_person: invalidating data will usually trigger a refresh of that data with a new API call.
......@@ -282,7 +288,11 @@ const mapDispatchToProps = (dispatch) => {
!> **The function that will be used to dispatch the action associated with fetching the data for `propName` must be identified by `api.propName` in the `mapDispatchToProps` function.**
That's all :confetti_ball:.
<div id="customGettersApi"></div>
?> :information_desk_person: All the subfunctions you define in `mapDispatchToProps` under the `api` key will be given one object parameter you may want to use. It is composed as follow: `{props, params}`. `props` will be an object corresponding to the `props` of the *connected* component. `params` is built with the `getQueryParams` and `getEndPointAttr` you may wish to override in your connected components. **`params` shall be *passed on* for filtering to work correctly**.
That's all :confetti_ball:.
### Words on updating data on the API
......
......@@ -146,7 +146,7 @@ const mapStateToProps = (state) => {
return {
countries: state.api.countriesAll,
currencies: state.api.currenciesAll,
serverModerationStatus: state.api.serverModerationStatusSpecific,
serverModerationStatus: state.api.serverModerationStatusAll,
};
};
......@@ -155,7 +155,7 @@ const mapDispatchToProps = (dispatch) => {
api: {
countries: () => dispatch(getActions("countries").readAll()),
currencies: () => dispatch(getActions("currencies").readAll()),
serverModerationStatus: () => dispatch(getActions("serverModerationStatus").readSpecific("")), // not needed for server moderation status
serverModerationStatus: () => dispatch(getActions("serverModerationStatus").readAll()), // not needed for server moderation status
},
};
};
......
......@@ -44,18 +44,30 @@ const mapStateToProps = (state) => {
*/
class CustomComponentForAPI extends Component {
customErrorHandlers = {}
// __apiAttr should be an object
// mapping the prop that needs to fetched with extra api attributes
// mapping should be : props_key => other_props_that contains the attribute to use
__apiAttr = null;
// __apiParam should be an object
// mapping the prop that needs to be fetched to the parameter to be passed to the read function
__apiParam = null;
/**
* A function with a propName parameter that should return the query parameters to add when reading that propName
*/
// eslint-disable-next-line no-unused-vars
getQueryParams = (propName) => undefined;
/**
* A function with a propName parameter that should return the endPoint attrs to add when reading that propName
*/
// eslint-disable-next-line no-unused-vars
getEndPointAttr = (propName) => undefined;
constructor(props) {
super(props);
if (typeof this.getEndPointAttr !== "function") {
throw new Error("getEndPointAttr misconfigured: it must be a function");
}
if (typeof this.getQueryParams !== "function") {
throw new Error("getQueryParams misconfigured: it must be a function");
}
// a little bit of optimization
// Stores the list of props that use the API
this.apiProps = Array();
......@@ -185,13 +197,21 @@ class CustomComponentForAPI extends Component {
* @memberof CustomComponentForAPI
*/
performReadFromApi(propName) {
if (this.__apiAttr && this.__apiAttr[propName]) {
this.props.api[propName](this.props[this.__apiAttr[propName]]);
} else if (this.__apiParam && this.__apiParam[propName]) {
this.props.api[propName](this.__apiParam[propName]);
} else {
this.props.api[propName]();
let endPointAttr = "",
queryParams = {};
const tmpQueryParams = this.getQueryParams(propName);
if (tmpQueryParams) {
queryParams = tmpQueryParams;
}
const tmpEndPointAttr = this.getEndPointAttr(propName);
if (tmpEndPointAttr) {
endPointAttr = tmpEndPointAttr;
}
this.props.api[propName]({ params: { queryParams, endPointAttr }, props: this.props });
}
/**
......
......@@ -154,7 +154,7 @@ class Editor extends Component {
performSave(data) {
this.notifyIsSaving();
this.props.saveData(
Object.assign({ __apiAttr: this.props.__apiAttr }, data),
data,
(newData) => this.handleSaveRequestWasSuccessful(newData)
);
}
......@@ -367,7 +367,6 @@ Editor.propTypes = {
classes: PropTypes.object.isRequired,
open: PropTypes.bool.isRequired, // should the editor be opened
subscribeToModuleWrapper: PropTypes.func.isRequired,
__apiAttr: PropTypes.oneOf([PropTypes.number, PropTypes.string, ""]),
rawModelData: PropTypes.object.isRequired,
closeEditorPanel: PropTypes.func.isRequired,
// props added in subclasses but are absolutely required to handle redux
......@@ -384,7 +383,6 @@ Editor.propTypes = {
Editor.defaultProps = {
open: false,
__apiAttr: "",
// eslint-disable-next-line no-console
closeEditorPanel: () => console.error("Dev forgot something...")
};
......
......@@ -28,6 +28,11 @@ function renderCore(rawModelData, classes, outsideData) {
}
class CountryDri extends Module {
/**
* @override
*/
getQueryParams = (propName) => propName === "countryDri" ? ({ countries: this.props.countryId }) : undefined;
customRender() {
const countryDriItems = this.getOnlyReadData("countryDri"),
{ countries, classes } = this.props,
......@@ -42,7 +47,6 @@ class CountryDri extends Module {
defaultModelData={{ countries: [this.props.countryId], importance_level: "-" }}
propsForEditor={{
outsideData: outsideData,
__apiAttr: this.props.countryId,
}}
>
{
......@@ -56,7 +60,6 @@ class CountryDri extends Module {
coreClasses={classes}
outsideData={outsideData}
moduleInGroupInfos={{ isInGroup: true, invalidateGroup: () => this.props.invalidateData() }}
__apiAttr={this.props.countryId}
/>
))
}
......@@ -64,8 +67,6 @@ class CountryDri extends Module {
);
}
__apiAttr = { countryDri: "countryId" };
}
CountryDri.propTypes = {
......@@ -77,16 +78,16 @@ CountryDri.propTypes = {
const mapStateToProps = (state) => {
return {
countryDri: state.api.countryDriSpecific,
countryDri: state.api.countryDriAll,
};
};
const mapDispatchToProps = (dispatch) => {
return {
api: {
countryDri: (countryId) => dispatch(getActions("countryDri").readSpecific(countryId)),
countryDri: ({ params }) => dispatch(getActions("countryDri").readAll(params)),
},
invalidateData: () => dispatch(getActions("countryDri").setInvalidatedSpecific(true))
invalidateData: () => dispatch(getActions("countryDri").setInvalidatedAll(true))
};
};
......
......@@ -43,6 +43,12 @@ function renderCore(rawModelData, classes, outsideData) {
class CountryScholarships extends Module {
/**
* @override
*/
getQueryParams = (propName) => propName === "countryScholarships" ? ({ countries: this.props.countryId }) : undefined;
customRender() {
const countryScholarshipsItems = this.getOnlyReadData("countryScholarships"),
{ countries, currencies, classes } = this.props,
......@@ -60,7 +66,6 @@ class CountryScholarships extends Module {
defaultModelData={{ countries: [this.props.countryId], importance_level: "-", currency: "EUR", obj_moderation_level: 0 }}
propsForEditor={{
outsideData: outsideData,
__apiAttr: this.props.countryId,
}}
>
{
......@@ -74,7 +79,6 @@ class CountryScholarships extends Module {
coreClasses={classes}
outsideData={outsideData}
moduleInGroupInfos={{ isInGroup: true, invalidateGroup: () => this.props.invalidateData() }}
__apiAttr={this.props.countryId}
/>
))
}
......@@ -82,8 +86,6 @@ class CountryScholarships extends Module {
);
}
__apiAttr = { countryScholarships: "countryId" };
}
CountryScholarships.propTypes = {
......@@ -97,16 +99,16 @@ CountryScholarships.propTypes = {
const mapStateToProps = (state) => {
return {
countryScholarships: state.api.countryScholarshipsSpecific,
countryScholarships: state.api.countryScholarshipsAll,
};
};
const mapDispatchToProps = (dispatch) => {
return {
api: {
countryScholarships: (countryId) => dispatch(getActions("countryScholarships").readSpecific(countryId)),
countryScholarships: ({ params }) => dispatch(getActions("countryScholarships").readAll(params)),
},
invalidateData: () => dispatch(getActions("countryScholarships").setInvalidatedSpecific(true))
invalidateData: () => dispatch(getActions("countryScholarships").setInvalidatedAll(true))
};
};
......
......@@ -32,10 +32,15 @@ function renderCore(rawModelData, classes, outsideData) {
class UniversityDri extends Module {
/**
* @override
*/
getQueryParams = (propName) => propName === "universityDri" ? ({ universities: this.props.univId }) : undefined;
customRender() {
const { universities, classes } = this.props,
outsideData = { universities },
univDriItems = this.getLatestReadData("universityDri", false);
univDriItems = this.getLatestReadData("universityDri");
return (
<ModuleGroupWrapper
......@@ -46,7 +51,6 @@ class UniversityDri extends Module {
defaultModelData={{ universities: [this.props.univId], importance_level: "-" }}
propsForEditor={{
outsideData: outsideData,
__apiAttr: this.props.univId,
}}
>
{
......@@ -60,7 +64,6 @@ class UniversityDri extends Module {
coreClasses={classes}
outsideData={outsideData}
moduleInGroupInfos={{ isInGroup: true, invalidateGroup: () => this.props.invalidateData() }}
__apiAttr={this.props.univId}
/>
))
}
......@@ -68,8 +71,6 @@ class UniversityDri extends Module {
);
}
__apiAttr = { universityDri: "univId" };
}
UniversityDri.propTypes = {
......@@ -81,16 +82,16 @@ UniversityDri.propTypes = {
const mapStateToProps = (state) => {
return {
universityDri: state.api.universityDriSpecific,
universityDri: state.api.universityDriAll,
};
};
const mapDispatchToProps = (dispatch) => {
return {
api: {
universityDri: (univId) => dispatch(getActions("universityDri").readSpecific(univId)),
universityDri: ({ params }) => dispatch(getActions("universityDri").readAll(params)),
},
invalidateData: () => dispatch(getActions("universityDri").setInvalidatedSpecific(true))
invalidateData: () => dispatch(getActions("universityDri").setInvalidatedAll(true))
};
};
......
......@@ -76,8 +76,6 @@ class UniversityGeneral extends Module {
/>
);
}
__apiAttr = { university: "univId" };
}
UniversityGeneral.propTypes = {
......@@ -97,7 +95,7 @@ const mapStateToProps = (state) => {
const mapDispatchToProps = (dispatch) => {
return {
api: {
university: (univId) => dispatch(getActions("universities").readSpecific(univId)),
university: ({ props }) => dispatch(getActions("universities").readSpecific(props.univId)),
},
invalidateData: () => dispatch(getActions("universities").setInvalidatedSpecific(true))
};
......
......@@ -38,6 +38,12 @@ function renderCore(rawModelData, classes, outsideData) {
}
class UniversityScholarships extends Module {
/**
* @override
*/
getQueryParams = (propName) => propName === "universityScholarships" ? ({ universities: this.props.univId, }) : undefined;
customRender() {
const univScholarshipsItems = this.getOnlyReadData("universityScholarships"),
{ universities, currencies, classes } = this.props,
......@@ -55,7 +61,6 @@ class UniversityScholarships extends Module {
defaultModelData={{ universities: [this.props.univId], importance_level: "-", currency: "EUR", obj_moderation_level: 0 }}
propsForEditor={{
outsideData: outsideData,
__apiAttr: this.props.univId,
}}
>
{
......@@ -69,7 +74,6 @@ class UniversityScholarships extends Module {
coreClasses={classes}
outsideData={outsideData}
moduleInGroupInfos={{ isInGroup: true, invalidateGroup: () => this.props.invalidateData() }}
__apiAttr={this.props.univId}
/>
))
}
......@@ -77,8 +81,6 @@ class UniversityScholarships extends Module {
);
}
__apiAttr = { universityScholarships: "univId" };
}
UniversityScholarships.propTypes = {
......@@ -91,16 +93,16 @@ UniversityScholarships.propTypes = {
const mapStateToProps = (state) => {
return {
universityScholarships: state.api.universityScholarshipsSpecific,
universityScholarships: state.api.universityScholarshipsAll,
};
};
const mapDispatchToProps = (dispatch) => {
return {
api: {
universityScholarships: (univId) => dispatch(getActions("universityScholarships").readSpecific(univId)),
universityScholarships: ({ params }) => dispatch(getActions("universityScholarships").readAll(params)),
},
invalidateData: () => dispatch(getActions("universityScholarships").setInvalidatedSpecific(true))
invalidateData: () => dispatch(getActions("universityScholarships").setInvalidatedAll(true))
};
};
......
......@@ -117,8 +117,6 @@ class UniversitySemestersDates extends Module {
/>
);
}
__apiAttr = { universitySemestersDates: "univId" };
}
......@@ -138,7 +136,7 @@ const mapStateToProps = (state) => {
const mapDispatchToProps = (dispatch) => {
return {
api: {
universitySemestersDates: (univId) => dispatch(getActions("universitiesSemestersDates").readSpecific(univId)),
universitySemestersDates: ({ props }) => dispatch(getActions("universitiesSemestersDates").readSpecific(props.univId)),
},
invalidateData: () => dispatch(getActions("universitiesSemestersDates").setInvalidatedSpecific(true))
};
......
......@@ -40,14 +40,16 @@ class History extends CustomComponentForAPI {
versionInView: 1
}
constructor(props) {
super(props);
const { contentTypeId, id } = props.modelInfo;
// set the __apiAttr here so that we can read the correct data
this.__apiParam = {
versions: { contentTypeId, id }
};
/**
* @override
*/
getEndPointAttr = (propName) => {
if (propName === "versions") {
const { contentTypeId, id } = this.props.modelInfo;
return `${contentTypeId}/${id}`;
} else {
return undefined;
}
}
/**
......@@ -242,16 +244,16 @@ History.defaultProps = {
const mapStateToProps = (state) => {
return {
versions: state.api.versionsSpecific
versions: state.api.versionsAll
};
};
const mapDispatchToProps = (dispatch) => {
return {
api: {
versions: (param) => dispatch(getActions("versions").readSpecific(`${param.contentTypeId}/${param.id}`)),
versions: ({ params }) => dispatch(getActions("versions").readAll(params)),
},
resetVersions: () => dispatch(getActions("versions").setInvalidatedSpecific(true)),
resetVersions: () => dispatch(getActions("versions").setInvalidatedAll(true)),
openFullScreenDialog: (innerNodes) => dispatch(openFullScreenDialog(innerNodes)),
closeFullScreenDialog: () => dispatch(closeFullScreenDialog()),
};
......
......@@ -106,7 +106,6 @@ ModuleGroupWrapper.propTypes = {
invalidateGroup: PropTypes.func.isRequired,
propsForEditor: PropTypes.shape({
outsideData: PropTypes.object,
__apiAttr: PropTypes.string,
}),
defaultModelData: PropTypes.object.isRequired, // to populate the fields of the form with default values
};
......
......@@ -199,7 +199,6 @@ class ModuleWrapper extends Component {
subscribeToModuleWrapper={(editorInstance) => this.editorInstance = editorInstance}
rawModelData={this.state.rawModelDataForEditor}
outsideData={this.props.outsideData}
__apiAttr={this.props.__apiAttr}
/>
{this.state.alert.open ?
<Alert
......@@ -275,7 +274,6 @@ class ModuleWrapper extends Component {
ModuleWrapper.defaultProps = {
buildTitle: () => null,
moduleInGroupInfos: { isInGroup: false, invalidateGroup: () => null },
__apiAttr: "",
};
ModuleWrapper.propTypes = {
......@@ -286,7 +284,6 @@ ModuleWrapper.propTypes = {
renderCore: PropTypes.func.isRequired,
coreClasses: PropTypes.object.isRequired,
outsideData: PropTypes.object,
__apiAttr: PropTypes.oneOfType([PropTypes.number, PropTypes.string]),
moduleInGroupInfos: PropTypes.shape({ isInGroup: PropTypes.bool.isRequired, invalidateGroup: PropTypes.func }).isRequired,
};
......
......@@ -28,14 +28,17 @@ import Loading from "../../../common/Loading";
* @extends {CustomComponentForAPI}
*/
class PendingModeration extends CustomComponentForAPI {
constructor(props) {
super(props);
const { contentTypeId, id } = props.modelInfo;
// set the __apiAttr here so that we can read the correct data
this.__apiParam = {
pendingModeration: { contentTypeId, id }
};
/**
* @override