- Updated the values of SECURE_BROWSER_XSS_FILTER, SECURE_CONTENT_TYPE_NOSNIFF...

- Updated the values of SECURE_BROWSER_XSS_FILTER, SECURE_CONTENT_TYPE_NOSNIFF and X_FRAME_OPTIONS settings according to the Django deployment checklist
- Moved the production_logs directory directly from the Django application directory (userbase/slub/usermerge) to the Git repository (userbase) to facilitate bulk modifications to the Django project (userbase/slub/slub) or application files, e.g. via update/replacement of the Django root directory (userbase/slub) itself, in production without affecting any already existing, independent production log files
- Ensured that the production_logs directory always exists in production to avoid raising FileNotFoundError exceptions when concurrent-log-handler attempts to output messages to the production log files
- Installed the mod_wsgi module in virtual environment context to enable project deployment with the Apache web server and included the relevant instructions in README.md
- Omitted the "UPDATING THE PROJECT" section and updated the "LIST OF SUGGESTED TO-DO ACTIONS" one in README.md
- Added the "INSTALLING AND CONFIGURING THE APACHE WEB SERVER TO DEPLOY THE PROJECT", "ACCESSING THE DJANGO ADMIN SITE OR RUNNING THE PROJECT APPLICATION (AFTER STARTING THE APACHE WEB SERVER)" and "CHECKLIST AND TIPS FOR PROJECT DEPLOYMENT" sections to README.md
- Made many project-deployment-specific (e.g. replaced "pip3 install virtualenv", "~/Desktop" and "pip install" with "sudo apt install virtualenv", "LOCAL_REPO_CONTAINER_DIR" and "$(which pip) install" respectively) as well as general corrections/enhancements (e.g. replaced "apt-get" and "/usr/bin/python3" with "apt" and "$(which python3)" respectively) to the instructions of README.md
- Made favicon.ico, apple_touch_icon.png, tick.png and user_report_creation.html non-executable (changed their permissions from 755 to 644) for enhanced security

.gitignore

@@ -8,5 +8,5 @@
 # virtual environment folder - see README.md
 venv/
 
-# folder that contains the log files created in production - see the "Logging" part of settings.py
-slub/usermerge/production_logs/
+# folder that contains the log files created in production - see the "Logging" section of settings.py
+production_logs/

requirements.txt

 concurrent-log-handler==0.9.12
 Django==2.0.4
+mod-wsgi==4.6.5
 mysqlclient==1.3.12
 pytz==2018.4

slub/slub/settings.py

@@ -8,6 +8,9 @@ https://docs.djangoproject.com/en/2.0/topics/settings/
 
 For the full list of settings and their values, see
 https://docs.djangoproject.com/en/2.0/ref/settings/
+
+For more information on deploying Django (with Apache and mod_wsgi), see
+https://docs.djangoproject.com/en/2.0/howto/deployment/ and README.md
 """
 
 import concurrent_log_handler, logging.config, os
@@ -70,6 +73,12 @@ TEMPLATES = [
 
 WSGI_APPLICATION = 'slub.wsgi.application'
 
+SECURE_BROWSER_XSS_FILTER = True
+
+SECURE_CONTENT_TYPE_NOSNIFF = True
+
+X_FRAME_OPTIONS = 'DENY'
+
 
 # Authentication
 # https://docs.djangoproject.com/en/2.0/topics/auth/
@@ -88,10 +97,18 @@ AUTHENTICATION_BACKENDS = ['django.contrib.auth.backends.ModelBackend', 'usermer
 #   - https://bugs.python.org/issue4749
 #   - The module provides an optional multi-threaded queue logging feature to perform logging in the background. The feature requires Python 3 to run and uses the standard QueueHandler and QueueListener logging classes to create background threads (one for each logger with configured handlers) that handle the logging, thus letting the foreground threads, i.e. those who initiate the logging, handle the requests quickly without blocking for write locks on the log files. It should be used after the logging configuration is set up. The latter is set up during the (usermerge) application setup, the application is set up each time it is loaded and, in production, it is loaded each time a child process is created by the main Apache process (see wsgi.py, https://github.com/django/django/blob/master/django/core/wsgi.py , https://github.com/django/django/blob/master/django/__init__.py and https://modwsgi.readthedocs.io/en/develop/user-guides/processes-and-threading.html). Since the logging configuration is set up for each child process created by the main Apache process, the feature would create too many background threads that could potentially interfere with the operation of Apache and mod_wsgi, which are responsible for managing processes and threads in production, in unexpected ways (https://groups.google.com/forum/#!topic/modwsgi/NMgrti4o9Bw). Therefore, it is recommended that the feature is NOT used, especially in production.
 # * Brief summary of the below defined configuration:
-#   - During development (the DEBUG setting is True), ALL log records/messages are output to the console/terminal (sys.stderr stream) where the Django development server has been run via the runserver command.
-#   - In production (the DEBUG setting is False), ALL log records/messages, EXCEPT FOR the DEBUG ones, are output to log files that exist in the production_logs directory of the usermerge application (directory). General INFO and WARNING messages are output to general_info_and_warnings.log, ERROR and CRITICAL messages are output to errors.log and INFO messages logged on the usermerge.views.recover_user_profile logger are output to changes_of_credentials_during_profile_recovery.log, while DEBUG messages are NOT output at all. Since the Django development server is inactive, NO messages are logged on the django.server logger for output to the sys.stderr stream. It is worth mentioning that mod_wsgi intercepts the sys.stdout and sys.stderr streams and redirects the output to the Apache error log (https://modwsgi.readthedocs.io/en/develop/user-guides/debugging-techniques.html).
+#   - If the DEBUG setting is True, i.e. during development, ALL log records/messages are output to the console/terminal (sys.stderr stream) where the Django development server has been run via the runserver command.
+#   - If the DEBUG setting is False, i.e. in production (and sometimes also during development to check the in-production functionality of the slub project without having to deploy it directly), ALL log records/messages, EXCEPT FOR the DEBUG ones, are output to log files that exist in the production_logs directory (this directory should exist exactly under the userbase Git repository to keep the existing production log files independent of any modifications to the project/application files). General INFO and WARNING messages are output to general_info_and_warnings.log, ERROR and CRITICAL messages are output to errors.log and INFO messages logged on the usermerge.views.recover_user_profile logger are output to changes_of_credentials_during_profile_recovery.log, while DEBUG messages are NOT output at all. Since the Django development server is inactive, NO messages are logged on the django.server logger for output to the sys.stderr stream. It is worth mentioning that mod_wsgi intercepts the sys.stdout and sys.stderr streams and redirects the output to the Apache error log (https://modwsgi.readthedocs.io/en/develop/user-guides/debugging-techniques.html).
 # * Detailed configuration and usage examples: https://www.webforefront.com/django/setupdjangologging.html
 
+PRODUCTION_LOGS_DIR = os.path.abspath(os.path.join(BASE_DIR, '..', 'production_logs/'))
+
+# Ensure that the production_logs directory exists in production (if it does not, create it) to avoid raising
+# "No such file or directory" FileNotFoundError exceptions when concurrent-log-handler attempts to output
+# messages to the production log files (https://stackoverflow.com/questions/32133856/logger-cannot-find-file).
+if not (DEBUG or os.path.exists(PRODUCTION_LOGS_DIR)):
+    os.mkdir(PRODUCTION_LOGS_DIR, 0o755) # http://blog.iqandreas.com/ubuntu/what-are-the-default-permissions-for-files-in-ubuntu/
+
 def LOG_LEVEL_IS_LOWER_THAN_ERROR(log_record):
     # This is the callback function of the exclude_errors log filter.
     # It is used to reject the ERROR and CRITICAL log records/messages that pass through the filter.
@@ -153,7 +170,7 @@ LOGGING = {
             'level': 'INFO',
             'filters': ['require_debug_false', 'exclude_errors'],
             'class': 'logging.handlers.ConcurrentRotatingFileHandler',
-            'filename': os.path.join(BASE_DIR, 'usermerge/production_logs/general_info_and_warnings.log'),
+            'filename': os.path.join(PRODUCTION_LOGS_DIR, 'general_info_and_warnings.log'),
             'maxBytes': 1024 * 100, # 100 KB
             'backupCount': 5,
             'formatter': 'verbose',
@@ -162,7 +179,7 @@ LOGGING = {
             'level': 'ERROR',
             'filters': ['require_debug_false'],
             'class': 'logging.handlers.ConcurrentRotatingFileHandler',
-            'filename': os.path.join(BASE_DIR, 'usermerge/production_logs/errors.log'),
+            'filename': os.path.join(PRODUCTION_LOGS_DIR, 'errors.log'),
             'maxBytes': 1024 * 200, # 200 KB
             'backupCount': 5,
             'formatter': 'verbose',
@@ -171,7 +188,7 @@ LOGGING = {
             'level': 'INFO',
             'filters': ['require_debug_false'],
             'class': 'logging.handlers.ConcurrentRotatingFileHandler',
-            'filename': os.path.join(BASE_DIR, 'usermerge/production_logs/changes_of_credentials_during_profile_recovery.log'),
+            'filename': os.path.join(PRODUCTION_LOGS_DIR, 'changes_of_credentials_during_profile_recovery.log'),
             'maxBytes': 1024 * 20, # 20 KB
             'backupCount': 5,
             'formatter': 'laconic',

slub/usermerge/models.py

@@ -5,7 +5,7 @@ from django.utils.timezone import localtime
 # Create your models here.
 # https://docs.djangoproject.com/en/2.0/ref/models/
 
-# The character set (encoding) of usermergeDB is set to utf8mb4 for proper Unicode support and the storage engine of its tables is set to InnoDB for proper transaction support. For more information on the database (table) options, e.g. character set, collation, SQL mode and storage engine, see README.md and the "Database" part of settings.py . It is worth mentioning that if MySQL is used as the database backend, the maximum length of CharFields and EmailFields with (unique) indexes in utf8mb4 encoding should be at most 191 characters (in utf8mb3 encoding, it should be at most 255 characters).
+# The character set (encoding) of usermergeDB is set to utf8mb4 for proper Unicode support and the storage engine of its tables is set to InnoDB for proper transaction support. For more information on the database (table) options, e.g. character set, collation, SQL mode and storage engine, see README.md and the "Database" section of settings.py . It is worth mentioning that if MySQL is used as the database backend, the maximum length of CharFields and EmailFields with (unique) indexes in utf8mb4 encoding should be at most 191 characters (in utf8mb3 encoding, it should be at most 255 characters).
 # The below defined models impose on fields only the absolutely necessary DB-related constraints. For more information on elaborate field constraints imposed outside the scope of models, e.g. minimum length constraint for username and password values, pattern constraints for ece_id and email values, prohibited clearance of registered ece_id values, etc., see forms.py and views.py .
 # Regarding the fields whose null option is set to True (any unique fields whose null option is undeclared, i.e. False by default, should NEVER be allowed to be empty): The empty DB value for DateTimeFields is NULL (interpreted as None in Python), thus designating the absence of value. The empty DB value for CharFields and EmailFields is the empty/zero-length string (''), which is a valid string value. For unique CharFields and EmailFields, such as ece_id and email of the User model, whose value may NOT always be declared when creating/updating the respective DB entries, this could lead in integrity errors ("Duplicate entry '' for key 'ece_id'/'email'"). To avoid such errors, NULL should be used as the empty DB value of these fields (the empty string should NOT be used in this case). For more information, see https://docs.djangoproject.com/en/2.0/ref/models/fields/ .
 # The password field of a user instance should NEVER contain the user's raw password. It should, instead, contain a hashed version of the latter created by the make_password() function for enhanced security (see examples in populateDB.py). The aforementioned function creates the hashed password using the PBKDF2 algorithm with a SHA256 hash by default. The length of the hashed password is 78 characters. For more information on password management, see https://docs.djangoproject.com/en/2.0/topics/auth/passwords/ .
@@ -18,7 +18,7 @@ from django.utils.timezone import localtime
 #   * The fields of a User instance should ALWAYS be in ONE of the following states: 1)ALL filled out, 2)ALL filled out EXCEPT FOR ece_id, 3)NONE filled out. The allowed transitions between field states are: 1-->1, 2-->1, 2-->2, 3-->1 and 3-->2 (empty field values can be changed to valid/consistent non-empty ones, non-empty field values can be changed validly/consistently but they should NEVER become empty). When a User instance is initially created, ALL its fields should be empty (state 3) and EXACTLY ONE Registry instance should be created to reference it - see the import_user_credentials() view. The fields can then be changed/filled out (state 1/2) ONLY by the corresponding user via the user profile edit form after he/she has logged in first - see the edit_user_profile() view (any other way of changing/filling out the fields is by NO means accepted).
 
 # A User instance retains ONE last_login value FOR EACH Platform instance that is associated with it via the corresponding Registry instance, while an Admin instance is associated ONLY with the SLUB "platform", i.e. it is NOT associated with any Platform instances, and thus retains EXACTLY ONE last_login value. Each time a user logs in, an appropriate user_logged_in signal is emitted - see the login() function in auth.py - causing the last_login field of the corresponding User/Admin instance for the login platform (SLUB in case of an Admin instance) to be updated INDEPENDENTLY of its other fields. A NON-empty User instance should ALWAYS retain a NON-empty last_login value FOR EACH of its associated Platform instances - see the edit_user_profile() and recover_user_profile() views. On the other hand, an empty User instance generally retains an empty last_login value for its ONLY associated Platform instance - see the import_user_credentials() view - but can also retain a NON-empty one for it (the user logs in and, although his/her profile is empty, he/she opts to log out - or is logged out automatically due to session expiration - WITHOUT filling it out). An Admin instance generally retains a NON-empty last_login value for the SLUB "platform" as it should be logged in to access any Admin-targeted functionality/view, but can also retain an empty one for it (the user has NEVER logged in but his/her profile has ALREADY been filled out in a NON-session context, e.g. fixture, interactive shell, file such as populateDB.py, etc. - see the relative references above).
-# Given that support for time zones is enabled (USE_TZ setting is True), DateTimeField values (datetime objects) are stored in UTC format in usermergeDB and are generally displayed in the current time zone, that is 'Europe/Athens' by default (specified by the TIME_ZONE setting), in templates. It is possible to manually convert these values to a different time zone or/and change their display format if needed, e.g. convert them to local time (default time zone, i.e. 'Europe/Athens') or/and display them in 'DD/MM/YYYY HH:MM:SS' format. For more information on time zone support and datetime objects, see https://docs.djangoproject.com/en/2.0/topics/i18n/timezones/ and the "Internationalization" part of settings.py .
+# Given that support for time zones is enabled (USE_TZ setting is True), DateTimeField values (datetime objects) are stored in UTC format in usermergeDB and are generally displayed in the current time zone, that is 'Europe/Athens' by default (specified by the TIME_ZONE setting), in templates. It is possible to manually convert these values to a different time zone or/and change their display format if needed, e.g. convert them to local time (default time zone, i.e. 'Europe/Athens') or/and display them in 'DD/MM/YYYY HH:MM:SS' format. For more information on time zone support and datetime objects, see https://docs.djangoproject.com/en/2.0/topics/i18n/timezones/ and the "Internationalization" section of settings.py .
 # The get_session_auth_hash() methods and is_authenticated attributes are needed for user authentication, e.g. in the login() and get_user() functions defined in auth.py and login_required() decorator used in views.py and decorators.py . Original source code for these methods/attributes can be found in https://github.com/django/django/blob/master/django/contrib/auth/base_user.py (for more information, see https://docs.djangoproject.com/en/2.0/topics/auth/customizing/).
 
 class User(models.Model):

slub/usermerge/views.py

@@ -22,14 +22,14 @@ from .models import Platform, Registry, User
 # Create your views here.
 # https://docs.djangoproject.com/en/2.0/topics/http/views/
 
-# For more information on configuring and using the logging system, see https://docs.djangoproject.com/en/2.0/topics/logging/ and the "Logging" part of settings.py .
+# For more information on configuring and using the logging system, see https://docs.djangoproject.com/en/2.0/topics/logging/ and the "Logging" section of settings.py .
 # For more information on default/customizing user authentication and password management, see https://docs.djangoproject.com/en/2.0/topics/auth/, auth.py, backends.py and AuthenticationMiddleware of middleware.py .
 # For more information on managing database transactions, see https://docs.djangoproject.com/en/2.0/topics/db/transactions/ .
 # For more information on making database queries, i.e. creating, retrieving, updating and deleting model instances, see https://docs.djangoproject.com/en/2.0/topics/db/queries/ .
 # For more information on working with forms, see https://docs.djangoproject.com/en/2.0/topics/forms/ .
 # For more information on handling file uploads, see https://docs.djangoproject.com/en/2.0/topics/http/file-uploads/ .
 # For more information on outputting CSV files, see https://docs.djangoproject.com/en/2.0/howto/outputting-csv/ .
-# For more information on time zone support and datetime objects, see https://docs.djangoproject.com/en/2.0/topics/i18n/timezones/ and the "Internationalization" part of settings.py .
+# For more information on time zone support and datetime objects, see https://docs.djangoproject.com/en/2.0/topics/i18n/timezones/ and the "Internationalization" section of settings.py .
 # For more information on using TemplateResponse objects - instead of standard HttpResponse objects and calls to the render() shortcut function - to display application pages/templates, see https://docs.djangoproject.com/en/2.0/ref/template-response/ .
 
 # The below defined views refer to the application site; therefore, they are not targeted at the default user model but at the application user models (see the _get_user_model_label() function of auth.py), i.e. the views can be accessed only by usermerge.{User/Admin} instances (the "usermerge." prefix is usually omitted in the application context) and not by auth.User ones - see the User_login_required() and Admin_login_required() decorators. Furthermore, the user instances (including the session one) that are created, retrieved, updated or deleted in the context of each view are usermerge.{User/Admin} instances. On the other hand, the admin site (see https://docs.djangoproject.com/en/2.0/ref/contrib/admin/) is not targeted at the application user models but at the default user model as only auth.User instances can be active staff members - each one has the is_active and is_staff attributes and the latter are set to True - and also have permissions - each one has the is_superuser, groups and user_permissions attributes as well as the relative methods, e.g. get_group_permissions(), get_all_permissions(), etc. - to access the site and utilize some or all of its capabilities. The admin site should generally be used for retrieving/reviewing model instances and not for creating, updating or deleting them - the latter actions should better be performed by appropriately defined views. When trying to create, update or delete instances via the admin site, extra care should be given to retain consistency in usermergeDB, e.g. a Platform instance could be created without any problem, while the updated password of an Admin instance would wrongly be stored in raw format (without being hashed first), etc. (for more information, see models.py).