User support for HTTP

In building the 7000 series of NAS appliances, we strove to create a solid storage product that’s revolutionary both in features and price/performance. This process frequently entailed rethinking old problems and finding new solutions that challenge the limitations of previous ones. Bill has a great example in making CIFS (SMB) a first-class data protocol on the appliance, from our management interface down to the kernel itself. I’ll discuss here the ways in which we’ve enhanced support for HTTP/WebDAV sharing, particularly as it coexists with other protocols like NFS, CIFS, and FTP.

WebDAV is a set of extensions to HTTP that allows clients (like web browsers) to treat web sites like filesystems. Windows, MacOS, and Gnome all have built-in WebDAV clients that allow users to “mount” WebDAV shares on their desktop and treat them like virtual disks. Since the client setup is about as simple as it could be (just enter a URL), this makes a great file sharing solution where performance is not critical and ease of setup is important. Users can simply click their desktop environment’s “connect to server” button and start editing files on the local company file server, where the data is backed up, automatically shared with their laptop, and even exported over NFS or CIFS as well.

User support

Many existing HTTP/WebDAV implementations consist of Apache and mod_dav. While this provides a simple, working implementation, its generality creates headaches when users want to share files over both HTTP and other protocols (like NFS). For one, HTTP allows the server to interpret users’ credentials (which are optional to begin with) however it chooses. This provides for enormous flexibility in building complex systems, but it means that you’ve got to do some work to make web users correspond to something meaningful in your environment (i.e., users in your company’s name service).

The other limitation of a basic Apache-based setup is that Apache itself has no support for assuming the identity of logged-in users. Typically, the web server runs as ‘webservd’ or ‘httpd’ or some other predefined system user. So in order for files to be accessible via the web, they must be accessible by this arbitrary user, which usually means making them world-accessible. Moreover, when files are created via the WebDAV interface, they end up being owned by the web server, rather than the actual user that created them.

By contrast, we’ve included strong support for system users in our HTTP stack. We use basic HTTP authentication to check a user’s credentials against the system’s name services, and then we process the request under their identity. The result is that proper filesystem permissions are enforced over HTTP, and newly created files are correctly owned by the user that created them, rather than the web server’s user. (This is not totally unlike a union of mod_auth_pam and mod_become, except that those are not very well supported.)

The user experience goes something like this: on my Mac laptop, I use the Finder’s “Connect to Server” option and enter the appliance’s WebDAV URL. I’m prompted for my username and password, which are checked against the local NIS directory server. Once in, my requests are handled by an httpd process which uses seteuid(2) to assume my identity. That means I can see exactly the same set of files I could see if I were using NFS, FTP, CIFS with identity mapping, etc. If I’m accessing someone else’s 644 file, then I can read but not write it. If I’m accessing my group’s 775 directory, then I can create files in it. It’s just as though I were using the local filesystem.

The mod_dav FAQ vaguely describes how one could do this, but implies that making it work requires introducing a huge security hole. Using Solaris’s fine-grained privileges, we give Apache worker processes just the proc_setid privilege (see privileges(5)). We don’t need httpd to run as root - we just need it to change among a set of unprivileged users. Any service expected to serve data on behalf of users must be granted this privilege – the NFS, CIFS, and FTP servers all do this (admittedly by running as root).

Of course, such a system is only as safe as its authentication and authorization mechanisms, and we’ve done our best to ensure that this code is safe and to mitigate the possible damage from potential exploits. It’s built on top of libpam and (of course) Solaris, so we know the foundation is solid.

Implementation notes

mod_user is our custom module which authenticates users and causes httpd to assume the identity of said users. It primarily consists of hooks into Apache’s request processing pipeline to authenticate users and change uid’s. We authenticate using pam(3PAM), which uses whatever name services have been set up for the appliance (NIS, LDAP, or locally created users).

Though mod_user itself is fairly simple, it’s also somewhat delicate from a security perspective. For example, since seteuid(2) changes the effective uid of the entire process, we must be sure that we’re never handling multiple requests concurrently. This is made pretty easy with the one-request-per-process threading module that is Apache’s default, but there’s still a bit of complexity around subrequests, where we may be running as some user other than Apache’s usual ‘webservd’ (since we’re processing a request for a particular user), but we need to authenticate the user as part of processing the subrequest. For local users, authentication requires reading our equivalent of /etc/shadow, which of course we can’t allow to be world-readable. But these kinds of issues are easily solved.

Other WebDAV enhancements

Enhancing the user model is one of a few updates we’ve made for HTTP sharing on the NAS appliance. I’ll soon discuss mod_dtrace, which facilitates HTTP Analytics by implementing a USDT provider for HTTP. We hope these sorts of features to make life better in many environments, whether you’re using WebDAV as a primary means of sharing files or you’re just giving read-only access to people’s home directories as a remote convenience.

Stay tuned to the Fishworks blogs for lots more discussion of Sun’s new line of storage appliances.