Hardening Apache

Gianluca writes "If security is not a concern, installing the Apache web server is a simple task even for an inexperienced system administrator. The problem is that security should always be a concern, and in case of Apache the information about making it secure can be sparse and fragmented. This is probably the reason why many web administrators are pretty clueless when it comes to Apache security. Needless to say, this creates a worrisome situation (to say the least): many web servers are vulnerable and exposed to thousands of potential attackers." Read on for Gianluca's review of Hardening Apache, a book intended to consolidate and clarify that information.

Hardening Apache fills a huge gap in this sense, providing web administrators with a complete and yet concise book aimed to guide them from the very beginning of the installation process to the final steps of the server configuration. The author, Tony Mobily, is also the mind behind Professional Apache Security, a book published by Wrox Press which I reviewed on Slashdot about 17 months ago. Since Wrox's unfortunate closure, some of the material from that book has been moved into Hardening Apache. More specifically:

• The excellent chapter on "jailing" Apache is exactly the same;
• The chapter on XSS attacks has been slightly improved;
• The chapter on logging, which was nothing remarkable, has been greatly improved. It now includes a complete architecture to log on a remote host using encryption and a TCP/IP connection.

The first chapter of the book deals with deploying a clean and safe base installation, which will then be the grounds for adding extra functionality. Unfortunately, this task is often underestimated. What I liked in this chapter is the step-by-step guide to correctly downloading the source distribution and verifying its integrity (by checking its digital signature), as well as the clean approach to the creation of a lean, easily readable configuration file, which grants a painless maintenance. A highlight of this section is the use of Nikto to analyse and explain common weaknesses and to show how to fix them.

Chapter 2 presents some vulnerabilities and explains how to exploit them. The chapter doesn't have any "pearls of wisdom" (but it's nevertheless important to show that Apache can be vulnerable), and presents some important reference sites every web administrator should be aware of.

Chapter 3 definitely deserves a special mention: after introducing the "common" ways of logging and syslogd's architecture, the author describes a rational approach to realizing a complete logging solution which entails remote log servers, encryption of logs, and the use of a MySQL database to better organize them.

Chapter 4 is the only one which deals with the "programming" side of web security. It is not a comprehensive guide on how to write safe programs for the web, as it focuses on cross-site scripting attacks; it shows how to secure a simple and vulnerable message board written in PHP.

The following chapter talks about security modules: it presents an interesting overview of the most useful modules related to security, which will help administrators understand the importance of third-party modules and explains how to install and use some of them. I also liked Chapter 6, which deals with the installation of Apache in a secure, chrooted environment: the chapter does a great job in guiding the reader through the non-trivial steps required to get Apache, Perl and PHP working correctly in such a restricted environment.

The last chapter presents a number of powerful and well-written scripts which anybody can use to automate security and keep an eye on their web server (monitoring log growth, Apache's responsiveness, and so on).

What's to like

Information throughout the book is very well focused and presented with a clean and friendly writing style. The book provides a clear and detailed walkthrough of the process of securing an Apache installation, covering both versions 1.3.x and 2.x and thus providing long lasting information. The book has lots of references and pointers to resources on the web, and - what's more important - instructions on how to read them. I also liked the "checkpoints" at the end of each chapter.

What's to consider

Apart from chapter 4 on cross-site scripting attacks, the book does not cover secure web programming at all. It doesn't cover OS hardening either, which is out of scope but part of the game anyway. Going through the book requires some familiarity with Unix and Apache; otherwise you will have to resort to other books for the very basic steps.

All in all, I found this sort of "new edition" of the book by Apress to be greatly enhanced, more homogeneous and better focused than the previous book: I had been happy with Wrox's version, but I am enthusiastic about this one. This is a book which should definitely be included in any serious Apache administrator's bookshelf.

---

You can purchase Hardening Apache from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

One of the unfortunate things about Apache...

Its configuration is unusually complex for a webserver. I wouldn't be surprised if many of its so-called "security holes" actually came about because of misconfiguration by an administrator who was confused by the layout of the documentation or config files.

In a way, Internet Information Services provides a more secure environment because an administrator gets a wealth of help and a decent initial configuration. In the end it's all about knowing your product, but it helps if the product helps you.

Re:One of the unfortunate things about Apache...

Sorry but IIS has more critical exploits in any given month than Apache has all YEAR! Why do you think Apache owns 70% of the web? Microsoft made a push in 2001 but could only get as high as 35%... at which point it tanked back to it's previous high of about 22%

Re:One of the unfortunate things about Apache...

Yep and I can still name 7 for every one of yours. How about Microsoft huh? Everytime a DDOS attack comes, gues who they duck and cover behind? LINUX and APACHE!!! :)

Do a check on Netcraft for *.microsoft.com sometimes and check the first couple hundred servers for what they are running. If IIS is so damn good, how come Microsoft is always duck and covering behind open source? :)

Re:One of the unfortunate things about Apache...

> In the end it's all about knowing your product,

In "Apache: The Definitive Guide", Ben and Peter Laurie suggest a way to learn how to build an Apache config file - start with a blank file, and start Apache. Oops, it won't start. OK, so what's it missing? Check the log files. It needs a User directive - OK, add that. Try to start. Hm, it started, but where do I put my HTML? Ah, add a DocumentRoot. And so forth.

This really doesn't take as long as it sounds - and after about 10 minutes of adding directives and restarting Apache, you'll have a lean configuration file that has just what you need in it - and you'll know how you got there, where the error logs are, and so on.

Re:One of the unfortunate things about Apache...

you'll also have a gimpy, crapped-up config file that other admins have to wade through after you get fired.

the best thing about the pre-built apache config files is the degree to which they are self-documenting. you can learn everything you need to know about apache just by reading through the comments. the second-best thing about them is that they're the same on all of your servers. the third best thing about them is that you don't have to read through a bunch of documentation to find out-of-date and non-working examples of config stanzas, because they're already there, commented out and waiting for some s/ action.

Re:One of the unfortunate things about Apache...

> a gimpy, crapped-up config file

Hm. I find smaller configuration files to have both a lower gimp quotient and a reduced crapification level.

> they are self-documenting.

True, although there's a ton of stuff in there most folks won't use - content negotiation and such-like.

> they're the same on all of your servers

Hm. Most of my servers have different config files - different vhost names, different modules, etc. Perhaps if they were all fronting one application that might be different...

Re:One of the unfortunate things about Apache...

Of course, you could reinterpret "start with a blank file" to mean "Add a '#' to the start of every line in the sample httpd.conf file". That would give you a "blank" file, but with all the documentation and examples very easily available.

Re:One of the unfortunate things about Apache...

I coulnd't agree more with you.

The point is that if you write your config file by hand, then you HAVE TO know each directive (well, sort of).
That's part of the reason why I wrote the first chapter that way...

Merc.
(The book's author)

Re:One of the unfortunate things about Apache...

In "Apache: The Definitive Guide", Ben and Peter Laurie suggest a way to learn how to build an Apache config file - start with a blank file, and start Apache. Oops, it won't start. OK, so what's it missing? Check the log files. It needs a User directive - OK, add that. Try to start. Hm, it started, but where do I put my HTML? Ah, add a DocumentRoot. And so forth.

No, its not add that. Its copy that and the associated comment from the original stock config file apache provides. Now your lean config file is documented, looks just like apache's and won't confuse your successor. I generally do this with any new daemon I install that requires a config file. Note that from a security standpoint this may not be good enough if the daemon's secure options are turned off by default since you may not see any warnings in the logs.

Re:One of the unfortunate things about Apache...

When I've used Apache2, it's configuration setup seems a lot more organized and structured. It's still text files, but they make more sense and they are modularized better.

Granted, that's just the in the distribution, not the server version... but at least it shows it's getting better at that. Now if only more people used Apache2, things would just fix themselves ;-)
-N

Re:One of the unfortunate things about Apache...

Its configuration is unusually complex for a webserver.

Really? I never found it all that hard. Most of the complex stuff is inserted in the default file, but is not insecure in of itself. You just have to think about everything you do after that.

I wouldn't be surprised if many of its so-called "security holes" actually came about because of misconfiguration by an administrator who was confused by the layout of the documentation or config files.

In my experience, most Apache security issues are related to giving access to files that shouldn't be on the web. Doing things like symlinking directories is very powerful, but it's also very dangerous. Also keep in mind that if an attacker can upload a script, he already has access to your system. These sorts of problems (as well as undiscovered buffer overflow issues) can be mitigated by the use of Jails. When you jail Apache, you ensure that any attacker that does gain access, will only have access to Apache files.

A typical Apache security problem would be something like an FTP server that has an anonymous account with access to a web-sharable directory. An attacker uploads a PHP script to the web-share, and it's game over for your machine/network.

secure

If security is not a concern, installing the Apache web server is a simple task even for an inexperienced system administrator Yet I still preffer IIS, according to research, it's more easy to install, and up to 70% more secure; according to research that is.

Don't forget

To Harden PHP [hardened-php.net] while you're at it.

Umm.

OpenBSD's [openbsd.org] Apache has a diff of 3 or 4 thousand lines over "stock" Apache. Why not just use that?

Re:Umm.

here is a link [google.com] about just that

and it's not that they haven't tried feeding the patches back, either. the ASF is being utterly retarded about accepting them. more [google.com] and more [google.com]

Interesting

One of the main things holding me back from hosting my own site rather than paying someone else to do the dirty work is the whole security issue. There are certain sercurity issues that I've never felt comfortable dealing with. Or rather I should say, being the "fall-guy" who has to deal with the inevitable attacks.

Maybe this book could be a first step for me?

Re:Interesting

Don't let not knowing about security hold you back from hosting your own site. Experiment, learn, have fun. Put an apache box up on a DMZ, put stupid content on it, see what happens. Look at your logs, see what's going on, learn from any mistakes you make along the way.

If you're in this industry, and are afraid to be the "fall guy" who has do deal with the inevitable attacks, or the "fall guy" in general, you'd better fasten your seat belt...you're in for a bumpy ride.

He who makes no mistakes makes nothing at all

Devil's Advocate

You could also take the time to read about Hardening IIS [microsoft.com]. Come get me Mods =)

Re:Devil's Advocate

We can also read The Bible [bible.org] if we believe in miracles.

Re:Devil's Advocate

A whole book on how to unplug a computer? Sheesh!

j/k

Is that anything like...

a cigar store wooden Indian? Sorry, I hadda say it...

Well

To harden Apache, you just rub it the right way.

*I apologize*

Re:Well

So that's where all those "child" processes come from . . .

Maximum Apache Security

Picked up Maximum Apache Security at Apache-Con last year and it has proven very useful. But any Apache administrator worth his salt knows most of this already. I don't see why you say it's fragmented and hard to find.

Fragmented Justification

The creators of Apache Server came up with the name due to it being based on a series of patches for httpd. "A Patchy Server." Get it? The name itself suggests its fragmented beginnings.

Stronghold

Does anyone know whether StrongHold [redhat.com] actually improves on Apache's security?

Don't advertise version number

I wish web servers wouldn't advertise their version number be default (e.g. in directory listings or HTTP headers). It's like giving an attacker an exact list of the exploits that will work on your server.

Great. Another Book of The Arcane.

There is a fundamental flaw with security hardening being in a separate book, sold by advertising and word of mouth, read separately and in a different medium than installation documentation, updated asynchronously, and expected to work. Would you accept a word processor with a separate book on "Master's Secrets on Keeping Your Word Processor From Crashing"?

With any luck, many hardening techniques will migrate towards the Apache installation process, or at least the Apache documentation.

just put an IIS server in front of it

you're apache install will stay safe because they'll be too distracted

Hardening apache?

Simple. Dip it in tree resin and let it fossilize. It should harden into amber in . . . oh, a few million years or so. Don't be impatient, though, or all you'll get is copal.

logging

For non-database logging, use mod_log_spread [backhand.org]. This also solves the problem of merging logs if you happen to run a web farm.

Thanks for posting. I could will look for it...

I do most of my work on a Domino server (say what you will, but its very secure and stable and I build customer apps really inexpensively) but Apache based servers (and they are myriad) keep intruding into my happy little world.

A year ago I wanted to put one public but found information on hardening it extremely limited -- or perhaps extremely disconnected.

If the book is indeed concise, it will be useful.

nice troll

Only an idiot couldn't figure out apache, and if you can't, you shouldn't be running web server at all. Exhibit A being your "more secure" IIS, which is full of more holes than swiss cheese.

thttpd

Apache is extremely complicated and therefore a potentail rats nest of potential security holes. Please don't get me wrong here: I like Apache very much for its great flexibility, yet I helped a lot of security aware companies to migrate to thttpd [acme.com] because they wanted a code base that could be scrutinized in a reasonable amount of time. Most web apps really don't need the flexibility of Apache anyway, and those who do, will have to be run in secure environments like, say, jails [freebsd.org] or other virtualized environments.

it should be secure out of the box

The following should be secure even for the most inexperienced user:

• Installing Apache
  
• Installing PHP
  
• Serving HTML files from the default Apache tree and/or ~/public_html
  
• Writing and serving PHP scripts that access the file system and databases with default permissions
  

If Apache and PHP can't fulfill those operations securely out of the box, then there is something wrong with either the design or implementation of Apache and PHP, not the experience level of the user.

In different words, the default should be secure, and users should have to go through extra steps to make it not secure.

OhioLinux & hardening apache

ohiolinux.org is having a speaker about hardening apache & apache2. and it looks as though its going to be a good feature packed lineup of oss speakers. hardening apache isn't exactly an easy task.. just don't think that you're going to be totally "secure." when you drop your defenses, you're at your weakest point. apache is a great server. and they have dominant market share. who says ms will win? 1 down, 100000 to go :)

If possible, use a different httpd

Apache is nice, and Apache is popular, but the simple fact that it's popular means that it attracts attention. If you don't need a module that's specifically for Apache, you can use a different webserver package (take a look at Freshmeat, there's a bunch) and you gain some security through obscurity.

Now, you still have to learn how to set up that package right, and keep up on updates, but diversity is a nice thing. Even if you just move some files (e.g. static content like images) off onto a side server, there's less to secure on the Apache box, so it's usually simpler.

Lack of security between users/virtual hosts

There is one thing that makes apache very unsecure. Suppose that you have few users each having it's own virtual host. Apache is running from exactly the same UID/GID for each virtual host! There is no way in pure apache to prevent user A from looking into user B vhost files (containing for example php scripts, password to sql databases etc).

You would need to run multiple apaches running from different UIDs for each user :/

That's bad. suexec it's not a option - it doesn't work with mod_python and other apache modules.

perchild MPM module that should do it doesn't work and apache developers are not interested in fixing that. No idea why.

metuxmpm MPM module was written instead perchild by external developers but it also doesn't work well unfortunately (for me it doesn't work at all).

NIce Defcon Presentation

http://www.bastille-linux.org/jay/Talks/slides-def con-securing-apache.pdf This helped me with a few extra ideas.

So why not do things differently?

Gee - you get this massive file to edit to get Apache up and running which is fine for some of geeky land (like the slash-dot crowd) but it creates a hurdle for your basic person who wants to have a running web site.

So why not create a smart program to generate the config file either as a script or as a GUI application that will have enough smarts to get your normal site up and running but with the possible security holes turned OFF and massive warnings written into the file so that when someone goes to edit it they have some clue provided.

Same for PHP.

Oh sure - it's soooo kewl to just edit the text file - and for the clued-in that's great. But something which can make a new safe file for you, AND check an existing file - seems like that's just smart.

Otherwise it's safety through obscurity - or insecurity through obscurity and neither of those make sense.

Puchasing Hardening Apache

"...You can purchase Hardening Apache from bn.com." Or, you can purchase it for $10 less from amazon.com

Re:I wonder...

You know... a recent study at MIT (I think) together with Simon and Garfunkel is showing that there's no difference between hardened and unhardened Apache.

broken downloads

This might be slightly off topic, but I have a question for all you Apache boys. I get this weird error from my Apache/2.0.48 (Win32) Server: it installs fine, I set it up properly, I set the passwords and the users, I test it, everything works fine, people can access and download stuff, then a few months later they start getting broken downloads. Meaning that, if they try to snag a 4.8 MB mp3 for instance, they end up being able to download only about 300 KB (the beginning of the file/song) after which it abruptly ends. Although the file clearly says 4.8 MB on the web page, Apache only lets you download some 3-400 KB. I have no idea what causes it (registry rot doesn't seem to be it) and the only way to fix it is to reinstall Apache - ten minutes later all is good again.
Any ideas, anyone?

Re:Stupid is a Stupid does

How many of you so called admins do this:

%su
%httpd start

when we all know it should be

%su
#httpd start

Re:Stupid is a Stupid does

only a farking moron disable's ping and BREAKS many RFC standards.

idiots disable ping. It there for a reason, just because asshat's use it to find targets does not make it something you dont need.

Re:Lead-in [OT, sorta]

but saying the least will always need less info.

And won't waste people's time (like mine)
who are going to react that your wasting people's time. Whoms reaction triggers more etc.

Ohhhh the horror.

the horror..

Re:Complexity

You could always pick up something like PHP Triad, which installs Apache, PHP, mySQL, etc ...
php Triad [sourceforge.net] ... I use it, and it works rather well just as a server to try on a home network

Re:Problems

Just use Apache 1.3 insted of 2.0 and it will stay up until you deside to take it down.

Re:Problems

Setting up a webserver, or any other system should ~not~ be trivial. Administrators and Systems Managers have their positions because they are well-versed in the platforms on which they work upon.

Oh, wait. I just kind of re-read your troll, um, i mean post. I don't think I have to say any more. I'll get back to my 7th hour of the day keeping it up. Apache, that is.

thats not been my experience

I designed the backend of www.babiesfirstchoice.com and we used apache 2. Its been hugely stable for us (the downtime we've seen was not due to apache problems) and lets us do everything we need to do on it. An IIS box would of cost thousands more due to licensing and the extra hardware needed to push M$ solutions (BFC currently runs on a athlon 1700xp with 512 megs of ddr and a basic ide hard drive, nothing fancy).

Re:Problems

I guess I can see how you would come to this.

I started using solely linux around 4 years ago. One of the first things I did was install apache and play around with that. I got it working with a little effort. After that I always built it from scratch. I eventually decided to try using apache out of the portage tree and it was set up as soon as I ran emerge and setup the configs in /etc/conf.d I've been using the gentoo build ever since then.

Anyways I had to develop some webapp and my boss wanted me to do it in php on IIS. We had a hell of a time getting all of the authentication and settings, we couldn't quite get it all working. We had trouble with ssl and a few other things. so my co-worker and I dropped apache on it and got it working with SSL in about 10 minutes.

I guess the short version of the story is: Hire MS sys admins for MS systems and here Linux sys admins for Linux systems. Rather than saying one is easier than the other, it might be better to say that one comes more naturally to a given user. And since I'm at a school [rose-hulman.edu] with a bunch of nerds and almost all of the people I know prefer to run linux, there's a good chance that we can probably get something like apache setup more easily than IIS.

Are we talking about the same Apache webserver?

There seemed to be little in the way of practical material that gave specific and step-by-step instructions for installing and running Apache on Linux.

Maybe you missed the "documentation" section at the apache.org website? Or, do a google search for "linux apache howto". Tons of good stuff out there.

Apache on Linux requires you to spend 8 hours per day just to keep it up and running,

On what planet is this true? There's about 4 things to change from one webserver to another; you build one config file for your environment, and for the next one modify the listen, the user if you want, the document_root, and maybe servlet mapping if you're using that. Trivial and one-time.

and while its performance and security is fine if you have the time and staff for it, there is no way to just set it up and let it sit while installing patches when needed.

Our experience differs profoundly. Perhaps someone like you needs to hire someone like me to help you get set up. It's a trivial setup, configuration is well documented, and once it's up and running a webserver doesn't need any attention whatsoever until the next version comes out or you decide you want to change what it does. Arguing against Apache on actual factual grounds would be one thing, but "it's hard to set up and lots of work to keep running" is demonstrably false.

Re:Problems

I'm wondering where you got that '8 hours per day' maintenance idea. Seriously, do you adjust your configuration for 8 hours a day, 5 days a week, year round, or what? What exactly requires your interaction with the system, once it's been set up? Occasional patching does not take 8 hours per day, every day. Ohh, and you do know that on unix you can remove a binary of a program that's running, put a new one in, and do a quick kill/start? Almost no downtime in that case, and there are tons of other ways to do it without any downtime whatsoever. Overall, I say troll.

Re:Problems

Ohh, and you do know that on unix you can remove a binary of a program that's running, put a new one in, and do a quick kill/start?

Actually, for most of the configuration changes (short of an actual version upgrade or SSL cert change), you can do an 'apachectl graceful' and it applies your changes to the _new_ sessions, while letting the existing sessions close in the natural flow of the users' use of your site. Nice for minor tweaks on the fly during the day, with zero downtime.

Re:Problems

WTF are you smoking, I certainly want to stay away from it. I've had my computer run apache for the webserver for years without having to change anything in the configurations (Mandrake default builds).

Re:Problems

Sigh, there are several ways to approach setting up an Apache server. All of them are easy.

First one is to start with an empty configuration file and then cut and past in portions of the standard file until you get a minimally working server.

The good part about this approach is that you get the least amount of bells and whistles added. Security via a small footprint is a good thing. The bad part about this approach is that you end up with a minimal server that may need more tweaking to get everything working as you need it.

The second approach is to take the original configuration file and start chopping things out of it. Test each deletion to make sure that everything you need still works. Use something as simple as RCS to keep track of your changes.

The good part about this approach is that you'll have a server until you break it. You will also have a nice record of every configuration change you've made. The bad part about this approach is that you may end up with a fatter server than you need. This violates a security maxim of making the least footprint on the net necessary to accomplish the task.

The third way to configure Apache is from scratch. This is somewhat more complex than the other two, and can lead to unmaintainable configuration files.

The bonuses for creating your own configuration file include understanding what goes on in the Apache configuration, and making a nice, modular configuration file. The bad part about this is that if you don't comment your file, you'll get an unmaintainable mess. Unfortunately some consultants think this is a good thing.

As for chrooting Apache, it took me less than 15 seconds via Google to find a step by step procedure http://www.faqs.org/docs/securing/chap29sec254.htm l [faqs.org] to chroot Apache on a Redhat Linux.