Apache server – Authentication – webdav

Apache server – Authentication – webdav

  • Access authorization with local authentication

    • Creating the password file (htpasswd)

    • User and group access management

    • Using external authentication system * Mysql * Ldap

    • Combination of external and local authentication

  • Webdav to have a place to store files

    • GNU / Linux webdav client

    • Secure your webdav

    • Secure your webdav for each operation

      • Limit the use of the webdav service

Access authorization with local authentication

We will now review our configuration for access to the directory admin, instead of using a system of limitation by IP address, we will implement a local authentication. The limitation by ip address is good, unfortunately when we are moving we do not always have the same ip address. We will use the mod_authn_core apache module.

Let’s start by validating that the latter is well loaded:

$ ls -l /etc/apache2/mods-enabled/*auth* lrwxrwxrwx 1 root root 33 Nov 25 08:20 /etc/apache2/mods-enabled/auth_basic.load -> ../mods-available/auth_basic.load lrwxrwxrwx 1 root root 33 Nov 25 08:20 /etc/apache2/mods-enabled/authn_core.load -> ../mods-available/authn_core.load lrwxrwxrwx 1 root root 33 Nov 25 08:20 /etc/apache2/mods-enabled/authn_file.load -> ../mods-available/authn_file.load lrwxrwxrwx 1 root root 33 Nov 25 08:20 /etc/apache2/mods-enabled/authz_core.load -> ../mods-available/authz_core.load lrwxrwxrwx 1 root root 33 Nov 25 08:20 /etc/apache2/mods-enabled/authz_host.load -> ../mods-available/authz_host.load lrwxrwxrwx 1 root root 33 Nov 25 08:20 /etc/apache2/mods-enabled/authz_user.load -> ../mods-available/authz_user.load

Comme nous pouvons le constater plusieurs module d’authentification sont chargé par défaut incluant modauthncore (ligne 2 😛 )

Pour information voici le contenu :

$ cat /etc/apache2/mods-enabled/authn_core.load LoadModule authn_core_module /usr/lib/apache2/modules/mod_authn_core.so

Apache loads a .so file that is the equivalent of a DLL that allows to have an additional instruction suite, available for the service. The module was compiled for the specific version of Apache it is not possible to take a module written for version 2.2 and make it work on version 2.4. We will see in more detail later Apache modules however it is good to have an understanding, summary of the operation.

Let’s modify the configuration file for our default site (/etc/apache2/sites-enabled/siteA-ssl.conf). Here is the configuration:

Options none AllowOverride None # Section d'authentification AuthType Basic AuthName "Authentication Required" AuthUserFile "/data/vhosts/siteA/privates/.htpasswd-admin" Require valid-user

Let’s analyze the parameters:

  • AuthType: Defines the type of authentication module, we saw that several modules were available in this case we will use the Basic mode. It is important to underline that in the current mode of communication the transfer of the password is transmitted in clear. In order to increase the level of security we should enable SSL support we will come back to it. Here are the types available:

  • AuthName: Message displayed to the user to enter the required authentication information.

  • AuthUserFile: Local file on the web server containing the list of users and password to validate user authentication.

  • Require: Instruction specifying the authorization criteria, in past demonstrations we used ip addresses as criteria here we can use usernames or define as in the present case any user that successfully authenticates.

Validate that there is no syntax error and reload the configuration file:

$ sudo apache2ctl configtest && sudo service apache2 restart Syntax OK * Restarting web server apache2 * ...done.

So if we go to the URL we have the message asking username and password.

However, since we never set username / password when we enter anything we have a 500 error indicating a problem on the server. The error message on the server is very clear:

$ sudo tail /var/log/apache2/error.log [Tue Dec 01 08:40:05.516453 2015] [authn_file:error] [pid 457:tid 2869857088] (2)No such file or directory: [client 172.17.42.1:52744] AH01620: Could not open password file: /data/vhosts/siteA/privates/.htpasswd-admin

Creating the password file (htpasswd)

To create the file containing the username and password we need the application htpasswd, this application is available in the package: apache2-utils

$ sudo apt-get install apache2-utils

We will create the file:

$ sudo htpasswd -c /data/vhosts/siteA/privates/.htpasswd-admin admin New password: Re-type new password: Adding password for user admin $ sudo htpasswd /data/vhosts/siteA/privates/.htpasswd-admin robert New password: Re-type new password: Adding password for user robert

IMPORTANT: between the 2 commands there is a difference, to create the file I used the option -c thereafter I do not use it. In fact it is important not that I use it otherwise it empties the file to create a new file, so we lose the information: – /. It’s dangerous, luckily you know what you’re doing: D, otherwise there are backups …

Let’s see the contents of the file, in both cases I put the same password (foo):

$ cat /data/vhosts/siteA/privates/.htpasswd-admin admin:$apr1$6gHZyTty$lxisZ66JPD7yK6K8fcXlU. robert:$apr1$fUI6se3h$40k/ftJPUtVOrptHlMiYB0

If we go back to the URL we should have a successful authentication with one or other of the users. The page indicating that we are in a “private section” should be displayed.

User and group access management

The above example is really the simplest method you with a password file, regardless of whether the user successfully establishes a connection to the password. On the other hand, if you have several sections that require the use of an authentication you probably would like to use 1 password file and define who to access in the apache configuration file.

If you know the name of the users you can change the following line with the name list:

 

Require valid-user

Here is the version with the specific names:

Require user admin thomas robert

This is fine but if you have a little work with this definition mode adding and deleting a user is boring as this is defined in the apache configuration or its extension via the .htaccess file it can not be delegated.

We will therefore define groups that may be edited by a third party or be reused in another context. We will create a text file with the definition of groups.

 

$ cat /data/vhosts/siteA/privates/htgroupe admins: admin robert superAdmins: admin thomas noobs: robert

We will define this group file and define which group can access the folder, here is the new configuration:

Options none AllowOverride None # Section d'authentification AuthType Basic AuthName "Authentication Required" AuthUserFile "/data/vhosts/siteA/privates/.htpasswd-admin" AuthGroupFile "/data/vhosts/siteA/privates/htgroupe" Require group admins

So we see 2 changes:

  • AuthGroupFile: this allows to define the group file which will be read with the information of the members

  • Require group admins: containing the group or groups that are allowed

Let’s now perform parsing and restart apache:

$ sudo apachectl configtest && sudo /etc/init.d/apache2 restart AH00526: Syntax error on line 33 of /etc/apache2/sites-enabled/siteA-ssl.conf: Invalid command 'AuthGroupFile', perhaps misspelled or defined by a module not included in the server configuration Action 'configtest' failed. The Apache error log may have more information.

OUPS an error: D, good 2 possibility:

  • the AuthGroupFile option is badly written or does not exist

  • It is missing a module …

I took the opportunity to show the importance and quality of the Apache site documentation.

TODO: add the screen shot for the module

Indeed, if we list the active modules and the available modules, we lack the module for managing the group file:

$ ls /etc/apache2/mods-enabled/*group* /etc/apache2/mods-available/*group* ls: cannot access /etc/apache2/mods-enabled/*group*: No such file or directory /etc/apache2/mods-available/authz_groupfile.load

Let’s activate it last and we’ll confirm that this works now:

$ cd /etc/apache2/mods-enabled $ sudo ln -s ../mods-available/authz_groupfile.load . $ sudo apachectl configtest && sudo /etc/init.d/apache2 restart Syntax OK * Restarting web server apache2 ...done.

We can now validate access: https://www.linux202-sitea.com/admin/

Using external authentication system

Mysql

Documentation : https://blog.froese.org/2014/06/13/authn-dbd-mysql-ubuntu-trusty/

Ldap

Documentation : https://httpd.apache.org/docs/2.4/mod/modauthnzldap.html

Combination of external and local authentication

Although we did not deal with the use of external authentication I would like to slip a word on the possibility of combining internal and external.

Here is an example of an authentication configuration using ldap authentication and file authentication:

 

<Directory "/www/docs/private"> AuthName "Private" AuthType Basic AuthBasicProvider file ldap AuthUserFile "/usr/local/apache/passwd/passwords" AuthLDAPURL ldap://ldaphost/o=yourorg Require valid-user

In this case, if the authentication with LDAP or from the password file is successful with a user, the latter will have access.

Référence :

  • http://httpd.apache.org/docs/current/fr/howto/auth.html

Webdav to have a place to store files

When training on rsnapshot I had mentioned using a webdav server to do the file transfer. Unfortunately we were not yet ready to set up Apache so we could not cover this feature. I did not have the primary objective to demonstrate and then following requests from participants here is an example of configuration. After all the training she is for YOU: D !!

The webdav protocol allows to perform file transfer, there are several clients for all platforms, we will see the possibility of mounting the file system under Linux. Let’s set up the configuration and then we will analyze the configuration together.

We are going to activate 2 modules mod_dav and mod_dav_fs, 2 methods to carry out the operation the realization of the links symbolic manually or the use of the command a2enmod.

 

$ sudo a2enmod dav Enabling module dav. To activate the new configuration, you need to run: service apache2 restart $ sudo a2enmod dav_fs Considering dependency dav for dav_fs: Module dav already enabled Enabling module dav_fs. To activate the new configuration, you need to run: service apache2 restart

Now we are going to change the configuration of siteA so that we can use the / webdav directory as the storage directory. Here is the content to add:

Dav On

WOwww magic: D, let’s all validate maybe I’m lying to you: P, that sounds too easy … Let’s create a file in the directory / data / vhosts / siteA / docroot / webdav /

$ cat /data/vhosts/siteA/docroot/webdav/toto super fichier dans webdav

If you access the URL: https://www.linux202-sitea.com/webdav/, you will have something that looks like this, you can click on the file toto and see its contents:

Secure your webdav

This is really interesting, but I doubt that you want to set up a free internet storage place for everyone to transfer data. You most likely want to be able to protect your data and those of your collaborators. I take this opportunity to emphasize the importance of performing the configuration with httpS communication so that when the username / password exchange they are encrypted.

In order to realize the demonstration of the establishment of the security I will make the creation of directory and file under the directory webdav:

 

$ cd /data/vhosts/siteA/docroot/webdav $ sudo mkdir -p commun devs marketing marketing/campagne_GPL marketing/campagne_FSF finance finance/super_secret $ sudo touch commun/activites_pour_nowel commun/le_footgolf devs/pourquoi_les_licences_c_important devs/les_bases_de_python marketing/campagne_GPL/copyleft_est_un_copyright marketing/campagne_GPL/la_licence_c_important marketing/campagne_FSF/Le_libre_pour_tous marketing/campagne_FSF/Le_libre_c_pour_TOI marketing/campagne_FSF/Le_libre_c_mieux marketing/campagne_future_a_planifier finance/benefice finance/client_prospect finance/client finance/super_secret/lettre_de_recouvrement_de_la_banque finance/super_secret/compte_au_panama

Which gives this:

$ ls -1R ./commun: activites_pour_nowel le_footgolf ./devs: les_bases_de_python pourquoi_les_licences_c_important ./finance: benefice client client_prospect super_secret ./finance/super_secret: compte_au_panama lettre_de_recouvrement_de_la_banque ./marketing: campagne_FSF campagne_GPL campagne_future_a_planifier ./marketing/campagne_FSF: Le_libre_c_mieux Le_libre_c_pour_TOI Le_libre_pour_tous ./marketing/campagne_GPL: copyleft_est_un_copyright la_licence_c_important

We will reuse what we learned earlier with Authentication Limitation and Group Access Management. Before setting up the limitation we will define our strategy by group:

  • all people authenticate can access the common section

  • the DEVS group can access the devs section

  • the FINANCE group can access the finance section but not the super_secret directory

  • the FINANCE_CA group can access the finance section and the super_secret directory

  • the MARKETING group can access the marketing section

We will create groups and users:

$ cat /data/vhosts/siteA/privates/.htpasswd-admin admin:$apr1$U8mylBZo$3aAVkmX.CkWpuwsx7/HT80 $ cat /data/vhosts/siteA/privates/htgroupe [ ... OUTPUT COUPÉ ... ] DEVS: bob robert bonny FINANCES: lee peter perry nesta FINANCES_CA: perry nesta MARKETING: dj-foot wailers $ LST_USERS=$(cat /data/vhosts/siteA/privates/htgroupe | grep ^[A-Z] | cut -d ":" -f 2 | tr -s " " "n" | sort | uniq ) $ for user in $LST_USERS ; do echo "le_mot_de_passe" | sudo htpasswd -i /data/vhosts/siteA/privates/.htpasswd-admin $user ; done Adding password for user bob Adding password for user bonny Adding password for user dj-foot Adding password for user lee Adding password for user nesta Adding password for user perry Adding password for user peter Adding password for user robert Adding password for user wailers $ cat .htpasswd-admin admin:$apr1$U8mylBZo$3aAVkmX.CkWpuwsx7/HT80 bob:$apr1$W.1roeGD$aLq8NNfz.XEpbrtZ.LEdu. bonny:$apr1$4BxK5D3a$ludP1cVjXYyJ5WN/5cyQa0 dj-foot:$apr1$BmNukD9R$mYXNIPMNgYHV2IwnRr/lO0 lee:$apr1$IobzpSiP$QAk5sntB/I/o5ebasxNBE. nesta:$apr1$WBLpPKWv$wu8AXh9xo4xNWUkkzzWHe0 perry:$apr1$DoWpviXH$Mt3iVvf/4/huSdumjnsqx0 peter:$apr1$SkCasVdW$4IgS2L3TvHPxdN4NUJICS1 robert:$apr1$YS.v9LIw$z/Uc4cA/REfZ1QqWUZlLI. wailers:$apr1$ZuDB1LPB$DYcrun7qhjyxY6mX326Fq/

Here we have now all of our created groups as well as our users with the same password but for the purposes of the demonstration no need for more. This also allows me to show how to automate via a script the creation / assignment of password with the command htpasswd.

Let’s set up the new configuration for the website and we’ll do some testing:

 

$ cat /etc/apache2/sites-enabled/siteA-ssl.conf [ ... OUTPUT COUPÉ ... ] ################ # Setup WEBDAV # ################ Dav On # Section d'authentification AuthType Basic AuthName "Authentication Required" AuthUserFile "/data/vhosts/siteA/privates/.htpasswd-admin" AuthGroupFile "/data/vhosts/siteA/privates/htgroupe" Require valid-user Require group DEVS Require group FINANCES Require group FINANCES_CA Require group MARKETING [ ... OUTPUT COUPÉ ... ]

This is the time to explain the content a bit, for the root of the webdav I defined the authentication with the password file and the one containing the list of group members. In this section I define that everyone can access through the Require valid-user statement. With this configuration, anyone who authenticates will have access to the files at the root of the webdav as well as all subdirectories !! Subsequently I set directory limitations, whether for devs, finances, etc. Each directory is limited by group, the advantage of this method is that there is no repetition of authentication instructions this helps visibility and management over time: D.

Let’s see now the result of this configuration, I will establish a connection with the user: bob is only a member of the DEVS group. Go to the URL: https://www.linux202-sitea.com/webdav/

Here is what we see:

 

 
Following the successful authentication we see the file titi and toto at the root as well as the common directory. Since the user is a member of the DEVS group he also sees the directory devs but NOT the other directories he does not have permissions!

 

If I perform the same operations with a GNU / Linux mount point, the view is a bit different

$ sudo mount.davfs https://www.linux202-sitea.com/webdav/ /mnt/dav/ Please enter the username to authenticate with server https://www.linux202-sitea.com/webdav/ or hit enter for none. Username: bob Please enter the password to authenticate user bob with server https://www.linux202-sitea.com/webdav/ or hit enter for none. Password: $ ls -l /mnt/dav/ total 2 drwxr-xr-x 2 root root 0 Jun 1 08:54 commun drwxr-xr-x 2 root root 0 Jun 1 08:53 devs drwxr-xr-x 2 root root 64 Jun 1 08:52 finance drwx------ 2 root root 0 May 31 17:08 lost+found drwxr-xr-x 2 root root 64 Jun 1 08:51 marketing -rw-r--r-- 1 x x 0 May 31 17:37 titi -rw-r--r-- 1 root root 26 May 31 08:46 toto $ ls -l /mnt/dav/finance/ total 0 $ ls -l /mnt/dav/devs/ total 0 -rw-r--r-- 1 root root 0 Jun 1 08:53 les_bases_de_python -rw-r--r-- 1 root root 0 Jun 1 08:53 pourquoi_les_licences_c_important $ ls -l /mnt/dav/marketing/ total 0

So when accessing with a mount point the directories at the root are visible but the content is empty! This is really a detail but it has to be taken into consideration depending on the situation. You will be able to test with other users …: D.

Secure your webdav for each operation

It is possible to go there in a more granular way, of course the more we go there in a granular way the management can become complicated. It is up to you to make the right choice and to be able to change your mind if you find that the first option was not the right one: D.

In the above example we restrict access by directory, however once the person to access the directory can perform all operations (Write, Delete, Copy, Read, …). As all the operations under realized through the Apache process it is not possible to limit the accesses via the file system. So how to provide read-only access and allow only certain type of operation.

TODO: change the links to have them in French

This is an opportunity to see another Limit and LimitExcept statement

Limit the use of the webdav service

We will be able to perform a limitation on the following HTTP requests:

  • General method of the HTTP protocol:

    • GET: This is the most common method for requesting a resource. A GET request has no effect on the resource, it must be possible to repeat the request without effect.

    • POST: This method is used to transmit data for processing to a resource (most often from an HTML form). The URI provided is the URI of a resource to which the sent data will apply. The result can be the creation of new resources or the modification of existing resources. Due to the improper implementation of HTTP (for Ajax) methods by some browsers (and the HTML standard that only supports GET and POST methods for forms), this method is often used instead of the PUT request, which should be used for updating resources.

    • PUT: This method replaces or adds a resource on the server. The URI provided is that of the resource in question.

    • DELETE: This method deletes a resource from the server.

    • CONNECT: This method allows to use a proxy as a communication tunnel

    • OPTIONS: This method provides the communication options for a resource or the server in general.

    • PATCH: This method allows, unlike PUT, to make a partial modification of a resource

  • Method specific to webdav protocol:

    • PROPFIND: This method allows the recovery of property, in XML format, from a server resource. This method also makes it possible to extract a structure group (also called hierarchical directory) from the server.

    • PROPPATCH: This method allows to modify and delete several properties of a resource with an action of type Atomic act.

    • MKCOL: This method allows directory creation.

    • COPY: This method copies a resource from one URI to another

    • MOVE: This method allows you to move a resource from one URI to another

    • LOCK: This method allows you to lock a resource, Webdav allows exclusive or shared locking.

    • UNLOCK: This method removes the lock of a resource.

So we can go there granularly on authorizations, 2 instructions are available:

  • Limit: Allows us to list the methods allowed by a user or a group.

  • LimitExcept: Allows us to define the methods NOT allowed so by default this is allowed.

In order to have the context in which this statement can be used:

 
p>Result if we wanted to have read-only access and full access for the dev section for example this will give:

 

$ cat /data/vhosts/siteA/privates/htgroupe [ ... OUTPUT COUPÉ ... ] DEVS: bob robert bonny DEVS_RO : nesta FINANCES: lee peter perry nesta [ ... OUTPUT COUPÉ ... ]

I realize the addition of the group DEVS_RO with as nesta member.

Configuring the virtual server:

[ ... OUTPUT COUPÉ ... ] Require group DEVS Require group DEVS_RO DEVS [ ... OUTPUT COUPÉ ... ]

Following the change you must reload the apache configuration. Using the client, making the mount point with the user who is a member of the read-only group:

$ sudo mount.davfs -o uid=x https://www.linux202-sitea.com/webdav/ /mnt/dav/ Please enter the username to authenticate with server https://www.linux202-sitea.com/webdav/ or hit enter for none. Username: nesta Please enter the password to authenticate user nesta with server https://www.linux202-sitea.com/webdav/ or hit enter for none. Password: $ ls -l /mnt/dav/ total 1 drwxr-xr-x 2 x root 0 Jun 1 08:54 commun drwxr-xr-x 2 x root 0 Jun 1 08:53 devs

If you try to create a file in the devs directories we will get an error message:

$ touch /mnt/dav/devs/un_fichier touch: setting times of ‘/mnt/dav/devs/un_fichier’: No such file or directory

If we look at the logs on the server we have the explicit message of the “problem”

$ tail /data/vhosts/siteA/logs/ssl_error.log [ ... OUTPUT COUPÉ ... ] [Thu Jun 09 08:48:52.731436 2016] [authz_core:error] [pid 115] [client 172.17.42.1:53514] AH01631: user nesta: authorization failure for "/webdav/devs/un_fichier":

Of course with a user in the right group we will not have this problem, you can now properly limit access to different directory.

Another interesting aspect of limitation will be the restriction of the size of the files that can be uploaded on the server you probably do not want a user uses your webdav server to exchange movies for example. So we will see the possibility of restricting the size of the files. The LimitRequestBody instruction provides the functionality to restrict the usable size.

 
ATTENTION: The size provided is in octects so if you want to allow a transfer of 2 Gig for example the value is astronomical 2147483647 :).

Let’s see an example of use, in my example I will limit to 100K it will be faster for the transfer time. Here is the apache configuration:

 

 

[ ... OUTPUT COUPÉ ... ] LimitRequestBody 102400 Require group DEVS Require group DEVS_RO DEVS [ ... OUTPUT COUPÉ ... ]

So we set the limit if I try to copy a file to the webdav server:

$ mount | grep dav https://www.linux202-sitea.com/webdav/ on /mnt/dav type fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=0,allow_other,max_read=16384,uid=1000,gid=0,helper=davfs) $ du -hs /tmp/access-denied.jpg 348K /tmp/access-denied.jpg $ cp /tmp/access-denied.jpg /mnt/dav/devs/test_fichier.jpg $ echo $? 0 $ du -hs /mnt/dav/devs/test_fichier.jpg 346K /mnt/dav/devs/test_fichier.jpg [ ... Quelque seconde plus tard ... ] $ du -hs /mnt/dav/devs/test_fichier.jpg du: cannot access ‘/mnt/dav/devs/test_fichier.jpg’: No such file or directory

If we look on the server logs:

$ tail -f /data/vhosts/siteA/logs/ssl_error.log [ ... OUTPUT COUPÉ ... ] [Fri Jun 10 08:40:00.051902 2016] [dav:error] [pid 66] (-102)Unknown error -102: [client 172.17.42.1:53868] An error occurred while reading the request body (URI: /webdav/devs/test_fichier.jpg) [500, #0] [ ... OUTPUT COUPÉ ... ]

Why during the original transfer the file was present then after it disappeared? The mount system to optimize access does not write directly but hides the operation. This is not always the ideal but when there is no problem this allows not to block the system the time of writing. Of course this forces your software to perform a post-write validation

  • Référence :

    • https://httpd.apache.org/docs/2.4/fr/mod/mod_dav.html