Disclaimer: The explanation provided on this website (Hotmaps Wiki) are indicative and for research purposes only. No responsibility is taken for the accuracy of the provided information, explanations and figures or for using them for unintended purposes.
Data privacy: By clicking OK below, you accept that this website may use cookies.
This page contains all information required for developers to contribute to the Hotmaps Platform or to understand how it is working.
Integration of new public datasets is handled as follow:
Data integration
If a dataset fails during integration, an issue is created on Taiga (project management platform). The issue shows the error raised and the developer should fix it and push again his work to Git so that the script can try to integrate it again the next evening.
The source code of the integration script is available at this link: Data integration
To add datasets on the list of public datasets, they must be pushed to a new Git repository on GitLab. Here is the GitLab Organization where datasets should be pushed: Datasets on GitLab.
Once a day the repositories are checked for new commits and integrated if so. The integration process checks whether the data conforms to the specification or not.
Here is the specification: Hotmaps_Data-upload-on-Gitlab_2017-12-04_V4.pdf
The code of the application is located on GitHub under the Hotmaps organization. This organization owns several repositories
If you want to add some code to Hotmaps you have two possibilities: if you want to update the interface or the backend directly, you have to modify the client or service repository of the toolbox. If you want to add your own calculation module, you can create your own repository by following the readme of the base_calculation_module repository
If you want to perform some work on the Git repository, please do not work directly with the master branch. Create a new branch from the develop branch, do your work on this branch and when your feature is tested, you can merge your work with the develop branch as shown in the following graph.
Git_workflow
In order to push something to some Hotmaps repository you have to be a member of the Hotmaps team, if you are not you are still able to perform a fork of our tool to develop your own tool.
You can find more information on how to work in these documents:
ReverseProxy_architecture_latest
All services and components are used through their own Docker container. All these containers are defined in a single docker-compose file. The image above represents the IT architecture of Hotmaps.
Some partner organizations limited communication to port 80 only. To avoid the problems caused by this limitation, the creation of a reverse proxy was made. This reverse proxy offers a single entry point and then distributes the request sent by the customer to the service concerned. The reverse proxy is composed of three components :
Three networks exist :
Each calculation module has its own Docker container.
Hotmaps uses Docker software and Docker-Compose tool to manage containers. A docker-compose.yml file contains the whole configuration of the Docker architecture (config. of containers, networks, links, ...). This allows containers to be run with one simple command :
docker-compose up
There is more about docker-compose on the website of Docker: Compose command-line reference and Compose file reference.
There is only one container that is run separately from others: it's the database because it needs to stay up all the time. That's why it's not in the docker-compose configuration file.
For the moment, the server is hosted at the HES-SO in Switzerland. There are 2 machines available: one for development (development and testing) and one for production (the actual toolbox, available at www.hotmaps.eu).
Both machines have the same specification:
We frequently run performance tests on the development server in order to guarantee a certain amount of concurrent users.
As an example, below are the results of the first beta release versus future release tests. The new release includes some performance improvements.
This example shows the performance tests of concurrent users using the same function: "duration curve for hectare selection". The bold line shows the limit where the server starts to raise errors. Hectare selection is a good example as it shows the queries that require the most resources.
Beta release of march 2019
Nb of simulated users | Average time | Median | Max time | Min time | Percentage of errors |
---|---|---|---|---|---|
1 | 1042 | 1042 | 1042 | 1042 | 0 |
20 | 3608 | 3858 | 4475 | 2156 | 0 |
50 | 7337 | 7595 | 11791 | 1568 | 0 |
100 | 13897 | 13644 | 24492 | 1875 | 0 |
200 | 27135 | 26380 | 51967 | 3045 | 0 |
300 | 41237 | 43083 | 73245 | 3223 | 10.67 |
Final release (August 2020)
Nb of simulated users | Average time | Median | Max time | Min time | Percentage of errors |
---|---|---|---|---|---|
1 | 1025 | 970 | 1151 | 956 | 0 |
20 | 1671 | 1732 | 2085 | 1113 | 0 |
50 | 3527 | 3622 | 5593 | 982 | 0 |
100 | 6648 | 6764 | 11854 | 1118 | 0 |
200 | 12904 | 12801 | 24163 | 1192 | 0 |
300 | 19521 | 19382 | 38116 | 1060 | 0 |
Daniel Hunacek, Lucien Zuber, Matthieu Dayer, in Hotmaps Wiki, Developers Section (September 2020)
This page was written by Daniel Hunacek, Lucien Zuber, and Matthieu Dayer (HES-SO).
☑ This page was reviewed by Mostafa Fallahnejad (EEG - TU Wien).
Copyright © 2016-2020: Daniel Hunacek, Lucien Zuber, and Matthieu Dayer
Creative Commons Attribution 4.0 International License
This work is licensed under a Creative Commons CC BY 4.0 International License.
SPDX-License-Identifier: CC-BY-4.0
License-Text: https://spdx.org/licenses/CC-BY-4.0.html
We would like to convey our deepest appreciation to the Horizon 2020 Hotmaps Project (Grant Agreement number 723677), which provided the funding to carry out the present investigation
View in another language:
Bulgarian* Czech* Danish* German* Greek* Spanish* Estonian* Finnish* French* Irish* Croatian* Hungarian* Italian* Lithuanian* Latvian* Maltese* Dutch* Polish* Portuguese (Portugal, Brazil)* Romanian* Slovak* Slovenian* Swedish*
* machine translated
Last edited by web, 2020-09-30 11:29:36