Skip to content

U
userbase
Commit 32efc5ac authored by Giorgos Kazelidis's avatar Giorgos Kazelidis
Browse files
Options

- Updated the values of SECURE_BROWSER_XSS_FILTER, SECURE_CONTENT_TYPE_NOSNIFF... 

- Updated the values of SECURE_BROWSER_XSS_FILTER, SECURE_CONTENT_TYPE_NOSNIFF and X_FRAME_OPTIONS settings according to the Django deployment checklist
- Moved the production_logs directory directly from the Django application directory (userbase/slub/usermerge) to the Git repository (userbase) to facilitate bulk modifications to the Django project (userbase/slub/slub) or application files, e.g. via update/replacement of the Django root directory (userbase/slub) itself, in production without affecting any already existing, independent production log files
- Ensured that the production_logs directory always exists in production to avoid raising FileNotFoundError exceptions when concurrent-log-handler attempts to output messages to the production log files
- Installed the mod_wsgi module in virtual environment context to enable project deployment with the Apache web server and included the relevant instructions in README.md
- Omitted the "UPDATING THE PROJECT" section and updated the "LIST OF SUGGESTED TO-DO ACTIONS" one in README.md
- Added the "INSTALLING AND CONFIGURING THE APACHE WEB SERVER TO DEPLOY THE PROJECT", "ACCESSING THE DJANGO ADMIN SITE OR RUNNING THE PROJECT APPLICATION (AFTER STARTING THE APACHE WEB SERVER)" and "CHECKLIST AND TIPS FOR PROJECT DEPLOYMENT" sections to README.md
- Made many project-deployment-specific (e.g. replaced "pip3 install virtualenv", "~/Desktop" and "pip install" with "sudo apt install virtualenv", "LOCAL_REPO_CONTAINER_DIR" and "$(which pip) install" respectively) as well as general corrections/enhancements (e.g. replaced "apt-get" and "/usr/bin/python3" with "apt" and "$(which python3)" respectively) to the instructions of README.md
- Made favicon.ico, apple_touch_icon.png, tick.png and user_report_creation.html non-executable (changed their permissions from 755 to 644) for enhanced security
parent 0320cadc
Show whitespace changes
Inline Side-by-side
Showing with 185 additions and 57 deletions
.gitignore
View file @ 32efc5ac
......@@ -8,5 +8,5 @@
# virtual environment folder - see README.md
venv/
# folder that contains the log files created in production - see the "Logging" part of settings.py
slub/usermerge/production_logs/
# folder that contains the log files created in production - see the "Logging" section of settings.py
production_logs/
README.md
View file @ 32efc5ac
......@@ -15,23 +15,25 @@
* Django version: 2.0.4 [https://www.djangoproject.com/]
* pytz version: 2018.4 [https://pypi.org/project/pytz/]
* concurrent-log-handler version: 0.9.12 [https://pypi.org/project/concurrent-log-handler/]
* mod_wsgi version: 4.6.5 [https://pypi.org/project/mod_wsgi/]
* mysqlclient version: 1.3.12 [https://pypi.org/project/mysqlclient/]
* MySQL server version: 5.7.24 [https://dev.mysql.com/]
* Apache web server version: 2.4.18 [https://httpd.apache.org/]
## DOWNLOADING THE PROJECT & SETTING UP THE ENVIRONMENT
(A) install pip3 (`pip3 --version` to see if it is already installed):
sudo apt-get install python3-pip
sudo apt install python3-pip
(B) install virtualenv (`virtualenv --version` to see if it is already installed):
pip3 install virtualenv
sudo apt install virtualenv
(C\) install and configure Git [sources:
      -- "How To Install Git with Apt" and "How To Set Up Git" sections of https://www.digitalocean.com/community/tutorials/how-to-install-git-on-ubuntu-16-04
      -- https://www.tutorialspoint.com/git/git_environment.htm
      ]:
      (1) install Git (`git --version` to see if it is already installed):
sudo apt-get install git
sudo apt install git
      (2) set the Git "name" and "email" global parameters (`git config --list` to see if they have already been set):
git config --global user.name "YOUR_NAME"
......@@ -39,9 +41,9 @@
(D) create a local instance of the remote repository (userbase) [sources:
      -- https://git-scm.com/docs/git-clone
      ]:
      (1) navigate to Desktop:
      (1) navigate to the directory (LOCAL_REPO_CONTAINER_DIR) where you would like to clone the repository (depending on the directory you chose to navigate to, you may need to run shell commands with `sudo` from here onwards to avoid raising "Permission Denied" exceptions):
cd ~/Desktop
cd LOCAL_REPO_CONTAINER_DIR
      (2) clone the repository to create the corresponding instance locally:
git clone https://git.softlab.ntua.gr/giorgkazelidis/userbase.git
......@@ -52,9 +54,11 @@
      -- steps 2 and 3 of https://help.dreamhost.com/hc/en-us/articles/215317948-How-to-install-Django-using-virtualenv
      -- step 3 of https://www.digitalocean.com/community/tutorials/how-to-create-a-django-app-and-connect-it-to-a-database
      -- end of section http://docs.python-guide.org/en/latest/dev/virtualenvs/#lower-level-virtualenv
      -- https://stackoverflow.com/questions/41060382/using-pip-to-install-packages-to-anaconda-environment
      -- https://docs.djangoproject.com/en/2.0/topics/install/
      -- https://pypi.org/project/mysqlclient/
      -- https://pypi.org/project/concurrent-log-handler/
      -- https://pypi.org/project/mod_wsgi/
      -- https://www.digitalocean.com/community/tutorials/how-to-install-mysql-on-ubuntu-16-04
      -- https://dev.mysql.com/doc/refman/5.7/en/mysql-tzinfo-to-sql.html
      -- https://datawookie.netlify.com/blog/2017/08/setting-up-time-zones-in-mysql/
......@@ -66,47 +70,44 @@
      -- https://docs.djangoproject.com/en/2.0/ref/django-admin/
      -- https://docs.python.org/3/tutorial/interpreter.html
      -- https://stackoverflow.com/questions/436198/what-is-an-alternative-to-execfile-in-python-3
      -- https://docs.python.org/3.6/tutorial/inputoutput.html#reading-and-writing-files
      -- https://docs.python.org/3.5/tutorial/inputoutput.html#reading-and-writing-files
      ]:
      (1) specify the full file path (/usr/bin/python3) to the installed Python version:
      (1) create the virtual environment (venv) and specify the desired Python version to be used:
which python3
      (2) create the virtual environment (venv) and specify the desired Python version to be used:
virtualenv ~/Desktop/userbase/venv -p /usr/bin/python3
      (3) activate the virtual environment:
virtualenv venv -p $(which python3)
      (2) activate the virtual environment:
source venv/bin/activate
      (4) install necessary Python and MySQL development headers and libraries (if you have not already done so):
      (3) install necessary Python and MySQL development headers and libraries (if you have not already done so):
sudo apt-get install python3-dev libmysqlclient-dev
      (5) install appropriate versions of virtual-environment-specific packages:
sudo apt install python3-dev libmysqlclient-dev
      (4) install appropriate versions of virtual-environment-specific packages:
pip install -r requirements.txt
            ALTERNATIVELY, you could install the packages (Django, mysqlclient, concurrent-log-handler and pytz - the last one is installed alongside Django) individually by typing the following:
$(which pip) install -r requirements.txt
            ALTERNATIVELY, you could install the packages (Django, mysqlclient, concurrent-log-handler, mod_wsgi and pytz - the last one is installed alongside Django) individually by running the following commands:
pip install Django
pip install mysqlclient
pip install concurrent-log-handler
      (6) install MySQL server (`mysqld --version` to see if it is already installed):
$(which pip) install Django==2.0.4
$(which pip) install mysqlclient==1.3.12
$(which pip) install concurrent-log-handler==0.9.12
$(which pip) install mod_wsgi==4.6.5
      (5) install MySQL server (`mysqld --version` to see if it is already installed):
            * ATTENTION: while installing MySQL server, you will be asked to set the password for the root user.
sudo apt-get install mysql-server
sudo apt install mysql-server
mysql_secure_installation
      (7) use the password that was set for the root user during MySQL server installation as the DATABASES['default']['PASSWORD'] value in **~/Desktop/userbase/slub/slub/settings.py**
      (6) use the password that was set for the root user during MySQL server installation as the DATABASES['default']['PASSWORD'] value in **LOCAL_REPO_CONTAINER_DIR/userbase/slub/slub/settings.py**
      (7) ensure that MySQL server is active by running `sudo systemctl status mysql` (if it is not, start it by running `sudo systemctl start mysql`) and `mysqladmin -u root -p version` (you will be asked to provide the password of the MySQL root user)
      (8) copy the time zone definitions of the system's zoneinfo database, which is located in the /usr/share/zoneinfo/ directory of Ubuntu and most other Unix-like systems, to the "mysql" database without worrying about some possible "Unable to load/Skipping" warnings (you will be asked to provide the password of the MySQL root user) and restart MySQL server (you can skip this substep if you have already executed it once):
mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root -p mysql
sudo service mysql restart
      (9) navigate to the project directory (slub):
sudo systemctl restart mysql
      (9) navigate to the Django project directory (slub):
cd slub
      (10) open MySQL console with root privileges:
python manage.py dbshell
            ALTERNATIVELY, you could run the following command by providing the password of the MySQL root user:
mysql -u root -p
            ALTERNATIVELY, you could run `mysql -u root -p` by providing the password of the MySQL root user
      (11) if the project database (usermergeDB) has already been created (`SHOW DATABASES;` to check), delete it:
DROP DATABASE usermergeDB;
......@@ -116,28 +117,28 @@
      (13) exit MySQL console:
EXIT;
      (14) use the collation that was selected during the project database creation as the DATABASES['default']['OPTIONS']['init_command']#collation_connection value in **~/Desktop/userbase/slub/slub/settings.py**
      (15) if MySQL server version is 5.5.4 and earlier, set SET_DEFAULT_STORAGE_ENGINE_TO_INNODB to "True" in **~/Desktop/userbase/slub/slub/settings.py**
      (14) use the collation that was selected during the project database creation as the DATABASES['default']['OPTIONS']['init_command']#collation_connection value in **LOCAL_REPO_CONTAINER_DIR/userbase/slub/slub/settings.py**
      (15) if MySQL server version is 5.5.4 and earlier, set SET_DEFAULT_STORAGE_ENGINE_TO_INNODB to "True" in **LOCAL_REPO_CONTAINER_DIR/userbase/slub/slub/settings.py**
      (16) initiate the project database (the tables are created during this substep):
python manage.py migrate
      (17) if you have previously set SET_DEFAULT_STORAGE_ENGINE_TO_INNODB to "True", set it back to "False"
      (18) populate the project database with test data by executing the populateDB.py script of the application directory (usermerge) via the Python interpreter:
      (18) populate the project database appropriately, e.g. by loading a Django fixture (see Django's `dumpdata` and `loaddata` administrative commands at https://docs.djangoproject.com/en/2.0/ref/django-admin/) or MySQL dump file (see https://dev.mysql.com/doc/refman/5.7/en/backup-and-recovery.html) into the database, by executing the populateDB.py script of the application directory (usermerge) via the Python interpreter (run the following command), etc.:
python manage.py shell --command="with open('./usermerge/populateDB.py') as popDB: exec(popDB.read())"
      (19) deactivate the virtual environment:
deactivate
## ACCESSING THE DJANGO ADMIN SITE OR RUNNING THE PROJECT (AFTER STARTING THE DJANGO DEVELOPMENT SERVER INSIDE THE VIRTUAL ENVIRONMENT)
(A) activate the virtual environment:
## ACCESSING THE DJANGO ADMIN SITE OR RUNNING THE PROJECT APPLICATION (AFTER STARTING THE DJANGO DEVELOPMENT SERVER INSIDE THE VIRTUAL ENVIRONMENT)
(A) activate the virtual environment of the local repository instance:
cd ~/Desktop/userbase
cd LOCAL_REPO_CONTAINER_DIR/userbase
source venv/bin/activate
(B) navigate to the project directory:
(B) navigate to the Django project directory:
cd slub
(C\) create a superuser account in the project database (usermergeDB) if you have not already done so (this step is necessary for accessing the Django admin site) [sources:
(C\) create a superuser account in the project database if you have not already done so (this step is necessary for accessing the Django admin site) [sources:
      -- https://docs.djangoproject.com/en/2.0/ref/contrib/admin/
      ]:
      * ATTENTION: while creating the superuser account, you will be asked to set the corresponding credentials (username and password) and (optional) email.
......@@ -146,25 +147,134 @@
(D) start the Django development server (at http://127.0.0.1:8000/):
python manage.py runserver
(E) access the Django admin site by navigating to http://127.0.0.1:8000/django-admin/ (via web browser) and filling out the emerging login form with the credentials of the aforementioned superuser account OR run the project by navigating to http://127.0.0.1:8000/
(E) access the Django admin site by navigating to http://127.0.0.1:8000/django-admin/ (via web browser) and filling out the emerging login form with the credentials of the aforementioned superuser account OR run the project application (usermerge) by navigating to http://127.0.0.1:8000/
## STOPPING THE DJANGO DEVELOPMENT SERVER (AND EXITING THE VIRTUAL ENVIRONMENT)
(A) stop the Django development server (running at http://127.0.0.1:8000/):
      -- return to the terminal where you have run `python manage.py runserver` and press `Ctrl-C`
(A) stop the Django development server by returning to the terminal where you have run `python manage.py runserver` and pressing `Ctrl-C`
(B) deactivate the virtual environment:
deactivate
## UPDATING THE PROJECT
Provided you have already downloaded the (source code of the) project (that is, you have followed the steps of the "DOWNLOADING THE PROJECT & SETTING UP THE ENVIRONMENT" section above), the recommended steps to update the (source code of the) project are:
      (1) delete the local instance of the remote repository (userbase) as well as the virtual environment:
## INSTALLING AND CONFIGURING THE APACHE WEB SERVER TO DEPLOY THE PROJECT
[sources:
      -- https://www.digitalocean.com/community/tutorials/how-to-install-the-apache-web-server-on-ubuntu-16-04
      -- https://pypi.org/project/mod_wsgi/
      -- https://www.digitalocean.com/community/tutorials/how-to-set-up-a-firewall-with-ufw-on-ubuntu-16-04
      -- https://docs.djangoproject.com/en/2.0/howto/deployment/wsgi/modwsgi/
      -- https://modwsgi.readthedocs.io/en/develop/user-guides/quick-configuration-guide.html
      -- https://www.digitalocean.com/community/questions/apache2-www-and-subsequent-folder-permissions
      -- https://askubuntu.com/questions/767504/permissions-problems-with-var-www-html-and-my-own-home-directory-for-a-website
      -- https://webmasters.stackexchange.com/questions/83633/have-disabled-apache-site-config-file-000-default-conf-but-it-still-seems-activ
]:
(A) install Apache web server and development package if you have not already done so (`apache2 -v` to see if server is already installed):
sudo apt install apache2 apache2-dev
(B) activate and adjust UFW firewall to open only port 80 for unencrypted incoming web traffic (`sudo ufw status verbose` to check the firewall status):
sudo ufw enable
sudo ufw allow 'Apache'
(C\) ensure that the web server is active by running `sudo systemctl status apache2` (if it is not, start it by running `sudo systemctl start apache2`) and navigating to http://SERVER_IP via web browser (`hostname -I` to specify SERVER_IP)
(D) navigate to the default VHost DocumentRoot of Apache (/var/www/html directory):
cd /var/www/html
(E) create the project deployment directory (softlab_userbase) that will contain the project document (DocumentRoot) and script directories to be deployed and navigate to it:
sudo mkdir softlab_userbase
cd softlab_userbase
(F) create the empty document directory (public_docs):
sudo mkdir public_docs
(G) follow all the steps of the "DOWNLOADING THE PROJECT & SETTING UP THE ENVIRONMENT" section above - at the end, a local instance of the remote repository (userbase) should have been created in the deployment directory and should serve as the script directory
(H) in **/var/www/html/softlab_userbase/userbase/slub/slub/settings.py**, replace the current value of SECRET_KEY with a similar, confidential one and ensure that DEBUG and ALLOWED_HOSTS are "False" and "['*']" respectively (if they are not, set them appropriately)
(I) navigate to the deployment directory:
cd /var/www/html/softlab_userbase
(J) change the user and group owners of the script directory to "www-data" (Apache system user/group) - `ls -l userbase` to verify the change:
sudo chown www-data:www-data userbase
(K) navigate to the Apache ServerRoot (/etc/apache2 directory):
cd /etc/apache2
(L) specify the Apache configuration directive needed to load the mod_wsgi module from the virtual environment (venv) of the script directory by running `mod_wsgi-express module-config` inside the aforementioned virtual environment and keeping the "LoadModule" line of the corresponding output:
source /var/www/html/softlab_userbase/userbase/venv/bin/activate
mod_wsgi-express module-config
deactivate
(M) use a text editor, e.g. gedit, and the specified module loading directive to create the mod_wsgi loading file (wsgi.load) in the module loading directory (mods-available):
sudo gedit mods-available/wsgi.load
with the following content:
# For more information, see the /var/www/html/softlab_userbase/userbase/README.md file as well as the following links:
# * https://httpd.apache.org/docs/current/mod/mod_so.html
# * https://pypi.org/project/mod_wsgi/
SET_THE_SPECIFIED_MODULE_LOADING_DIRECTIVE_HERE
(N) activate the mod_wsgi module loading and restart the web server (at http://SERVER_IP) to apply it:
sudo a2enmod wsgi
sudo systemctl restart apache2
(O) use a text editor, e.g. gedit, to create the project deployment configuration file (softlab_userbase.conf) in the VHost configuration directory (sites-available):
sudo gedit sites-available/softlab_userbase.conf
with the following content (make sure to replace SERVER_IP, PROCESSES_NUMBER, THREADS_NUMBER and python3.V with the appropriate values, e.g. run `hostname -I` to specify SERVER_IP and `python3 --version` to specify python3.V - in the current context, it is python3.5):
# For more information, see the /var/www/html/softlab_userbase/userbase/README.md and /etc/apache2/sites-available/000-default.conf files
# as well as the following links:
# * https://httpd.apache.org/docs/current/mod/core.html
# * https://httpd.apache.org/docs/current/logs.html
# * https://modwsgi.readthedocs.io/en/develop/configuration.html
# * https://modwsgi.readthedocs.io/en/develop/user-guides/quick-configuration-guide.html
# * https://modwsgi.readthedocs.io/en/develop/user-guides/virtual-environments.html
# * https://docs.djangoproject.com/en/2.0/howto/deployment/wsgi/modwsgi/
<VirtualHost *:80>
ServerName SERVER_IP
ServerAlias userbase.softlab.ntua.gr
ServerAdmin webmaster@courses.softlab.ntua.gr
DocumentRoot /var/www/html/softlab_userbase/public_docs
Alias /static/admin/ /var/www/html/softlab_userbase/userbase/venv/lib/python3.V/site-packages/django/contrib/admin/static/admin/
Alias /static/ /var/www/html/softlab_userbase/userbase/slub/usermerge/static/
WSGIDaemonProcess softlab_userbase processes=PROCESSES_NUMBER threads=THREADS_NUMBER display-name=%{GROUP} \
python-home=/var/www/html/softlab_userbase/userbase/venv python-path=/var/www/html/softlab_userbase/userbase/slub
WSGIProcessGroup softlab_userbase
WSGIApplicationGroup %{GLOBAL}
WSGIScriptAlias / /var/www/html/softlab_userbase/userbase/slub/slub/wsgi.py
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
(P\) activate the project deployment configuration and reload the web server to apply it:
sudo a2ensite softlab_userbase
sudo systemctl reload apache2
## ACCESSING THE DJANGO ADMIN SITE OR RUNNING THE PROJECT APPLICATION (AFTER STARTING THE APACHE WEB SERVER)
(A) create a superuser account in the project database inside the virtual environment of the project script directory if you have not already done so (this step is necessary for accessing the Django admin site) - see the "ACCESSING THE DJANGO ADMIN SITE OR RUNNING THE PROJECT APPLICATION (AFTER STARTING THE DJANGO DEVELOPMENT SERVER INSIDE THE VIRTUAL ENVIRONMENT)" section above for more information:
cd /var/www/html/softlab_userbase/userbase
source venv/bin/activate
python slub/manage.py createsuperuser
deactivate
(B) start the Apache web server (`sudo systemctl status apache2` to see if it is already active):
sudo systemctl start apache2
(C\) access the Django admin site by navigating to http://SERVER_IP/django-admin/ and filling out the emerging login form with the credentials of the aforementioned superuser account OR run the project application (usermerge) by navigating to http://SERVER_IP - the static files/directories of the project application and Django admin site are also accessible at http://SERVER_IP/static/ and http://SERVER_IP/static/admin/ respectively
rm -rf ~/Desktop/userbase
&nbsp; &nbsp; &nbsp; (2) follow steps (D) and (E) of the "DOWNLOADING THE PROJECT & SETTING UP THE ENVIRONMENT" section above
## CHECKLIST AND TIPS FOR PROJECT DEPLOYMENT
* https://docs.djangoproject.com/en/2.0/howto/deployment/
* https://www.freecodecamp.org/news/django-in-the-wild-tips-for-deployment-survival-9b491081c2e4/
* https://www.agileleaf.com/blog/a-better-way-to-manage-settings-py-in-your-django-projects/
* https://simpleisbetterthancomplex.com/tips/2017/07/03/django-tip-20-working-with-multiple-settings-modules.html
## LIST OF SUGGESTED TO-DO ACTIONS
* Create and use a Makefile for setting up/updating the project [https://www.gnu.org/software/make/manual/make.html]
* Create and use a criteria-based (e.g. last_login-based) batch deletion view for User/Registry instances to ensure that the project database (usermergeDB) remains "small" and thus efficient to interact with - see models.py
* Define a cron job that uses MySQL's `mysqldump` administrative command to regularly create project database backups (dump files) needed for recovery in case of system crashes, hardware failures, accidental data deletion, etc. [https://www.ostechnix.com/a-beginners-guide-to-cron-jobs/, https://dev.mysql.com/doc/refman/5.7/en/backup-and-recovery.html]
* Define a cron job that uses Django's `clearsessions` administrative command to regularly clean expired sessions out of the django_session table and prevent it from growing excessively in the project database [https://www.ostechnix.com/a-beginners-guide-to-cron-jobs/, https://docs.djangoproject.com/en/2.0/topics/http/sessions/#clearing-the-session-store]
* Create and use a criteria-based (e.g. last_login-based) batch deletion view for User/Registry instances to ensure that the usermerge_user and usermerge_registry tables do not grow excessively in the project database - see models.py
* Create and run unit tests [https://docs.djangoproject.com/en/2.0/topics/testing/]
* Make better use of the Django admin site capabilities
* Make better use of Django's model validation capabilities [https://docs.djangoproject.com/en/2.0/ref/models/instances/]
......
slub/slub/settings.py
View file @ 32efc5ac
......@@ -8,6 +8,9 @@ https://docs.djangoproject.com/en/2.0/topics/settings/
For the full list of settings and their values, see
https://docs.djangoproject.com/en/2.0/ref/settings/
For more information on deploying Django (with Apache and mod_wsgi), see
https://docs.djangoproject.com/en/2.0/howto/deployment/ and README.md
"""
import concurrent_log_handler, logging.config, os
......@@ -70,6 +73,12 @@ TEMPLATES = [
WSGI_APPLICATION = 'slub.wsgi.application'
SECURE_BROWSER_XSS_FILTER = True
SECURE_CONTENT_TYPE_NOSNIFF = True
X_FRAME_OPTIONS = 'DENY'
# Authentication
# https://docs.djangoproject.com/en/2.0/topics/auth/
......@@ -88,10 +97,18 @@ AUTHENTICATION_BACKENDS = ['django.contrib.auth.backends.ModelBackend', 'usermer
# - https://bugs.python.org/issue4749
# - The module provides an optional multi-threaded queue logging feature to perform logging in the background. The feature requires Python 3 to run and uses the standard QueueHandler and QueueListener logging classes to create background threads (one for each logger with configured handlers) that handle the logging, thus letting the foreground threads, i.e. those who initiate the logging, handle the requests quickly without blocking for write locks on the log files. It should be used after the logging configuration is set up. The latter is set up during the (usermerge) application setup, the application is set up each time it is loaded and, in production, it is loaded each time a child process is created by the main Apache process (see wsgi.py, https://github.com/django/django/blob/master/django/core/wsgi.py , https://github.com/django/django/blob/master/django/__init__.py and https://modwsgi.readthedocs.io/en/develop/user-guides/processes-and-threading.html). Since the logging configuration is set up for each child process created by the main Apache process, the feature would create too many background threads that could potentially interfere with the operation of Apache and mod_wsgi, which are responsible for managing processes and threads in production, in unexpected ways (https://groups.google.com/forum/#!topic/modwsgi/NMgrti4o9Bw). Therefore, it is recommended that the feature is NOT used, especially in production.
# * Brief summary of the below defined configuration:
# - During development (the DEBUG setting is True), ALL log records/messages are output to the console/terminal (sys.stderr stream) where the Django development server has been run via the runserver command.
# - In production (the DEBUG setting is False), ALL log records/messages, EXCEPT FOR the DEBUG ones, are output to log files that exist in the production_logs directory of the usermerge application (directory). General INFO and WARNING messages are output to general_info_and_warnings.log, ERROR and CRITICAL messages are output to errors.log and INFO messages logged on the usermerge.views.recover_user_profile logger are output to changes_of_credentials_during_profile_recovery.log, while DEBUG messages are NOT output at all. Since the Django development server is inactive, NO messages are logged on the django.server logger for output to the sys.stderr stream. It is worth mentioning that mod_wsgi intercepts the sys.stdout and sys.stderr streams and redirects the output to the Apache error log (https://modwsgi.readthedocs.io/en/develop/user-guides/debugging-techniques.html).
# - If the DEBUG setting is True, i.e. during development, ALL log records/messages are output to the console/terminal (sys.stderr stream) where the Django development server has been run via the runserver command.
# - If the DEBUG setting is False, i.e. in production (and sometimes also during development to check the in-production functionality of the slub project without having to deploy it directly), ALL log records/messages, EXCEPT FOR the DEBUG ones, are output to log files that exist in the production_logs directory (this directory should exist exactly under the userbase Git repository to keep the existing production log files independent of any modifications to the project/application files). General INFO and WARNING messages are output to general_info_and_warnings.log, ERROR and CRITICAL messages are output to errors.log and INFO messages logged on the usermerge.views.recover_user_profile logger are output to changes_of_credentials_during_profile_recovery.log, while DEBUG messages are NOT output at all. Since the Django development server is inactive, NO messages are logged on the django.server logger for output to the sys.stderr stream. It is worth mentioning that mod_wsgi intercepts the sys.stdout and sys.stderr streams and redirects the output to the Apache error log (https://modwsgi.readthedocs.io/en/develop/user-guides/debugging-techniques.html).
# * Detailed configuration and usage examples: https://www.webforefront.com/django/setupdjangologging.html
PRODUCTION_LOGS_DIR = os.path.abspath(os.path.join(BASE_DIR, '..', 'production_logs/'))
# Ensure that the production_logs directory exists in production (if it does not, create it) to avoid raising
# "No such file or directory" FileNotFoundError exceptions when concurrent-log-handler attempts to output
# messages to the production log files (https://stackoverflow.com/questions/32133856/logger-cannot-find-file).
if not (DEBUG or os.path.exists(PRODUCTION_LOGS_DIR)):
os.mkdir(PRODUCTION_LOGS_DIR, 0o755) # http://blog.iqandreas.com/ubuntu/what-are-the-default-permissions-for-files-in-ubuntu/
def LOG_LEVEL_IS_LOWER_THAN_ERROR(log_record):
# This is the callback function of the exclude_errors log filter.
# It is used to reject the ERROR and CRITICAL log records/messages that pass through the filter.
......@@ -153,7 +170,7 @@ LOGGING = {
'level': 'INFO',
'filters': ['require_debug_false', 'exclude_errors'],
'class': 'logging.handlers.ConcurrentRotatingFileHandler',
'filename': os.path.join(BASE_DIR, 'usermerge/production_logs/general_info_and_warnings.log'),
'filename': os.path.join(PRODUCTION_LOGS_DIR, 'general_info_and_warnings.log'),
'maxBytes': 1024 * 100, # 100 KB
'backupCount': 5,
'formatter': 'verbose',
......@@ -162,7 +179,7 @@ LOGGING = {
'level': 'ERROR',
'filters': ['require_debug_false'],
'class': 'logging.handlers.ConcurrentRotatingFileHandler',
'filename': os.path.join(BASE_DIR, 'usermerge/production_logs/errors.log'),
'filename': os.path.join(PRODUCTION_LOGS_DIR, 'errors.log'),
'maxBytes': 1024 * 200, # 200 KB
'backupCount': 5,
'formatter': 'verbose',
......@@ -171,7 +188,7 @@ LOGGING = {
'level': 'INFO',
'filters': ['require_debug_false'],
'class': 'logging.handlers.ConcurrentRotatingFileHandler',
'filename': os.path.join(BASE_DIR, 'usermerge/production_logs/changes_of_credentials_during_profile_recovery.log'),
'filename': os.path.join(PRODUCTION_LOGS_DIR, 'changes_of_credentials_during_profile_recovery.log'),
'maxBytes': 1024 * 20, # 20 KB
'backupCount': 5,
'formatter': 'laconic',
......
slub/usermerge/models.py
View file @ 32efc5ac
......@@ -5,7 +5,7 @@ from django.utils.timezone import localtime
# Create your models here.
# https://docs.djangoproject.com/en/2.0/ref/models/
# The character set (encoding) of usermergeDB is set to utf8mb4 for proper Unicode support and the storage engine of its tables is set to InnoDB for proper transaction support. For more information on the database (table) options, e.g. character set, collation, SQL mode and storage engine, see README.md and the "Database" part of settings.py . It is worth mentioning that if MySQL is used as the database backend, the maximum length of CharFields and EmailFields with (unique) indexes in utf8mb4 encoding should be at most 191 characters (in utf8mb3 encoding, it should be at most 255 characters).
# The character set (encoding) of usermergeDB is set to utf8mb4 for proper Unicode support and the storage engine of its tables is set to InnoDB for proper transaction support. For more information on the database (table) options, e.g. character set, collation, SQL mode and storage engine, see README.md and the "Database" section of settings.py . It is worth mentioning that if MySQL is used as the database backend, the maximum length of CharFields and EmailFields with (unique) indexes in utf8mb4 encoding should be at most 191 characters (in utf8mb3 encoding, it should be at most 255 characters).
# The below defined models impose on fields only the absolutely necessary DB-related constraints. For more information on elaborate field constraints imposed outside the scope of models, e.g. minimum length constraint for username and password values, pattern constraints for ece_id and email values, prohibited clearance of registered ece_id values, etc., see forms.py and views.py .
# Regarding the fields whose null option is set to True (any unique fields whose null option is undeclared, i.e. False by default, should NEVER be allowed to be empty): The empty DB value for DateTimeFields is NULL (interpreted as None in Python), thus designating the absence of value. The empty DB value for CharFields and EmailFields is the empty/zero-length string (''), which is a valid string value. For unique CharFields and EmailFields, such as ece_id and email of the User model, whose value may NOT always be declared when creating/updating the respective DB entries, this could lead in integrity errors ("Duplicate entry '' for key 'ece_id'/'email'"). To avoid such errors, NULL should be used as the empty DB value of these fields (the empty string should NOT be used in this case). For more information, see https://docs.djangoproject.com/en/2.0/ref/models/fields/ .
# The password field of a user instance should NEVER contain the user's raw password. It should, instead, contain a hashed version of the latter created by the make_password() function for enhanced security (see examples in populateDB.py). The aforementioned function creates the hashed password using the PBKDF2 algorithm with a SHA256 hash by default. The length of the hashed password is 78 characters. For more information on password management, see https://docs.djangoproject.com/en/2.0/topics/auth/passwords/ .
......@@ -18,7 +18,7 @@ from django.utils.timezone import localtime
# * The fields of a User instance should ALWAYS be in ONE of the following states: 1)ALL filled out, 2)ALL filled out EXCEPT FOR ece_id, 3)NONE filled out. The allowed transitions between field states are: 1-->1, 2-->1, 2-->2, 3-->1 and 3-->2 (empty field values can be changed to valid/consistent non-empty ones, non-empty field values can be changed validly/consistently but they should NEVER become empty). When a User instance is initially created, ALL its fields should be empty (state 3) and EXACTLY ONE Registry instance should be created to reference it - see the import_user_credentials() view. The fields can then be changed/filled out (state 1/2) ONLY by the corresponding user via the user profile edit form after he/she has logged in first - see the edit_user_profile() view (any other way of changing/filling out the fields is by NO means accepted).
# A User instance retains ONE last_login value FOR EACH Platform instance that is associated with it via the corresponding Registry instance, while an Admin instance is associated ONLY with the SLUB "platform", i.e. it is NOT associated with any Platform instances, and thus retains EXACTLY ONE last_login value. Each time a user logs in, an appropriate user_logged_in signal is emitted - see the login() function in auth.py - causing the last_login field of the corresponding User/Admin instance for the login platform (SLUB in case of an Admin instance) to be updated INDEPENDENTLY of its other fields. A NON-empty User instance should ALWAYS retain a NON-empty last_login value FOR EACH of its associated Platform instances - see the edit_user_profile() and recover_user_profile() views. On the other hand, an empty User instance generally retains an empty last_login value for its ONLY associated Platform instance - see the import_user_credentials() view - but can also retain a NON-empty one for it (the user logs in and, although his/her profile is empty, he/she opts to log out - or is logged out automatically due to session expiration - WITHOUT filling it out). An Admin instance generally retains a NON-empty last_login value for the SLUB "platform" as it should be logged in to access any Admin-targeted functionality/view, but can also retain an empty one for it (the user has NEVER logged in but his/her profile has ALREADY been filled out in a NON-session context, e.g. fixture, interactive shell, file such as populateDB.py, etc. - see the relative references above).
# Given that support for time zones is enabled (USE_TZ setting is True), DateTimeField values (datetime objects) are stored in UTC format in usermergeDB and are generally displayed in the current time zone, that is 'Europe/Athens' by default (specified by the TIME_ZONE setting), in templates. It is possible to manually convert these values to a different time zone or/and change their display format if needed, e.g. convert them to local time (default time zone, i.e. 'Europe/Athens') or/and display them in 'DD/MM/YYYY HH:MM:SS' format. For more information on time zone support and datetime objects, see https://docs.djangoproject.com/en/2.0/topics/i18n/timezones/ and the "Internationalization" part of settings.py .
# Given that support for time zones is enabled (USE_TZ setting is True), DateTimeField values (datetime objects) are stored in UTC format in usermergeDB and are generally displayed in the current time zone, that is 'Europe/Athens' by default (specified by the TIME_ZONE setting), in templates. It is possible to manually convert these values to a different time zone or/and change their display format if needed, e.g. convert them to local time (default time zone, i.e. 'Europe/Athens') or/and display them in 'DD/MM/YYYY HH:MM:SS' format. For more information on time zone support and datetime objects, see https://docs.djangoproject.com/en/2.0/topics/i18n/timezones/ and the "Internationalization" section of settings.py .
# The get_session_auth_hash() methods and is_authenticated attributes are needed for user authentication, e.g. in the login() and get_user() functions defined in auth.py and login_required() decorator used in views.py and decorators.py . Original source code for these methods/attributes can be found in https://github.com/django/django/blob/master/django/contrib/auth/base_user.py (for more information, see https://docs.djangoproject.com/en/2.0/topics/auth/customizing/).
class User(models.Model):
......
slub/usermerge/static/images/apple_touch_icon.png 100755 → 100644
View file @ 32efc5ac
File mode changed from 100755 to 100644
slub/usermerge/static/images/favicon.ico 100755 → 100644
View file @ 32efc5ac
File mode changed from 100755 to 100644
slub/usermerge/static/images/tick.png 100755 → 100644
View file @ 32efc5ac
File mode changed from 100755 to 100644
slub/usermerge/templates/user_report_creation.html 100755 → 100644
View file @ 32efc5ac
File mode changed from 100755 to 100644
slub/usermerge/views.py
View file @ 32efc5ac
......@@ -22,14 +22,14 @@ from .models import Platform, Registry, User
# Create your views here.
# https://docs.djangoproject.com/en/2.0/topics/http/views/
# For more information on configuring and using the logging system, see https://docs.djangoproject.com/en/2.0/topics/logging/ and the "Logging" part of settings.py .
# For more information on configuring and using the logging system, see https://docs.djangoproject.com/en/2.0/topics/logging/ and the "Logging" section of settings.py .
# For more information on default/customizing user authentication and password management, see https://docs.djangoproject.com/en/2.0/topics/auth/, auth.py, backends.py and AuthenticationMiddleware of middleware.py .
# For more information on managing database transactions, see https://docs.djangoproject.com/en/2.0/topics/db/transactions/ .
# For more information on making database queries, i.e. creating, retrieving, updating and deleting model instances, see https://docs.djangoproject.com/en/2.0/topics/db/queries/ .
# For more information on working with forms, see https://docs.djangoproject.com/en/2.0/topics/forms/ .
# For more information on handling file uploads, see https://docs.djangoproject.com/en/2.0/topics/http/file-uploads/ .
# For more information on outputting CSV files, see https://docs.djangoproject.com/en/2.0/howto/outputting-csv/ .
# For more information on time zone support and datetime objects, see https://docs.djangoproject.com/en/2.0/topics/i18n/timezones/ and the "Internationalization" part of settings.py .
# For more information on time zone support and datetime objects, see https://docs.djangoproject.com/en/2.0/topics/i18n/timezones/ and the "Internationalization" section of settings.py .
# For more information on using TemplateResponse objects - instead of standard HttpResponse objects and calls to the render() shortcut function - to display application pages/templates, see https://docs.djangoproject.com/en/2.0/ref/template-response/ .
# The below defined views refer to the application site; therefore, they are not targeted at the default user model but at the application user models (see the _get_user_model_label() function of auth.py), i.e. the views can be accessed only by usermerge.{User/Admin} instances (the "usermerge." prefix is usually omitted in the application context) and not by auth.User ones - see the User_login_required() and Admin_login_required() decorators. Furthermore, the user instances (including the session one) that are created, retrieved, updated or deleted in the context of each view are usermerge.{User/Admin} instances. On the other hand, the admin site (see https://docs.djangoproject.com/en/2.0/ref/contrib/admin/) is not targeted at the application user models but at the default user model as only auth.User instances can be active staff members - each one has the is_active and is_staff attributes and the latter are set to True - and also have permissions - each one has the is_superuser, groups and user_permissions attributes as well as the relative methods, e.g. get_group_permissions(), get_all_permissions(), etc. - to access the site and utilize some or all of its capabilities. The admin site should generally be used for retrieving/reviewing model instances and not for creating, updating or deleting them - the latter actions should better be performed by appropriately defined views. When trying to create, update or delete instances via the admin site, extra care should be given to retain consistency in usermergeDB, e.g. a Platform instance could be created without any problem, while the updated password of an Admin instance would wrongly be stored in raw format (without being hashed first), etc. (for more information, see models.py).
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment