Backdoor Targeting Apache Servers Spreads To Nginx, Lighttpd 136
An anonymous reader writes "Last week's revelation of the existence of Linux/Cdorked.A, a highly advanced and stealthy Apache backdoor used to drive traffic from legitimate compromised sites to malicious websites carrying Blackhole exploit packs, was only the beginning — ESET's continuing investigation has now revealed that the backdoor also infects sites running the nginx and Lighttpd webservers. Researchers have, so far, detected more than 400 webservers infected with the backdoor, and 50 of them are among the world's most popular and visited websites." Here's the researchers' original report.
I have a stupid question. (Score:1, Interesting)
Why do the developers force us to tell the world what engine we use?
Re:I have a stupid question. (Score:5, Interesting)
Are you refering to the http headers that identify the server version? If so then yes, it is a stupid question since, every webserver which I have ever configured has had an option to turn that off. Not that I ever bothered, if it was so useful, it would be turned off by default.
Fingerprinting doesn't take that long, especially for well known services. Might be of some use if you really to run something obscure. In any case, even if they don't know if you are vulnerable, how long does it take to find out? Little use there.
Re: (Score:2)
Is G-Wan affected ? (Score:2)
I have some site running lighttpd, others I run G-Wan
Is G-Wan affected ?
Thanks in advance for any tips that you can share with us.
Thanks again !!
Re: (Score:2)
Checker code: download, compile, run (Score:2)
Thanks for the info !!
Looks like I ain't gonna enjoy lots of sleep from now until next weekend
You could download and compile (for your web server) the detection C code provided here [welivesecurity.com]. Then you'll have less uncertainty.
I had to cross-compile it for an old Synology box with a PowerPC 8241 processor; it seems to be clean.
Re: (Score:2)
Hey, great tool, thanks.
But I wouldn't trust a positive result that much. If it says your computer is clean, it probably is, if it says it isn't, you'd better take another look at it before formating.
Re: (Score:1)
Not delivering traffic is standard behaviour for G-Wan; it doesn't need a backdoor or virus for that.
Re:I have a stupid question. (Score:4, Informative)
Quite frankly, I don't think the webserver was the entry point for Cdorkd.A since as far as I read it was mainly machines with cpanel that were infected. Even if the problem wasn't cpanel Apache doesn't run with the right permissions to change it's own binary. If the entry point is elsewhere, once they are in the machine with root access discovering what web server software being used is trivial.
Rather than worrying about something as trivial as the web server software, I would be much more concerned about why none of the control panels I've come across seem to have any sort of secure design. They run as root without any sort of privilege separation and edit the config files even when daemons are available that have a database back end.
Re: (Score:1)
Re:I have a stupid question. (Score:5, Funny)
What kind of developer thinks that a web server needs a GUI?
Where else are they going to put the ON and OFF buttons?
Re:I have a stupid question. (Score:5, Insightful)
Implemented an idea poorly does not make it a bad idea.
Re:I have a stupid question. (Score:4, Interesting)
> since you clearly cannot give them root access.
and yet that's what it seems to be doing here. I heard a lot of folks say that LXC was DOA, because it didn't offer any protection against the classic "escalate chrooted root user to full system access," and I am not an expert but I'd say that has changed, you _can_ give your customers root without giving them root on the host system. Check out http://docker.io/ [docker.io] </shameless>
(I heard there were alternatives to docker too, but I haven't found any other than RTFM and Edit The Damn Configs And Cross Your Fingers. Docker has just entered version 0.3 release and development is moving quickly.)
Re: (Score:2)
Re: (Score:2)
No, I got it, I was just chiming in chorus with the other folks that were saying, "this tool does not need to be run as root. why does it have such a large attack surface? it's very expensive and widely deployed so why is it that it's always been such crap"
I don't run cpanel myself (I understand it's quite expensive to license) but I have been a customer of cpanel hosts before, I totally get it, it's meant to limit your access while providing you with administrative capabilities that are above the bare mi
Re: (Score:2)
> Apache doesn't run with the right permissions to change it's own binary
However it runs with enough permission to execute an exploit that would elevate access. The further you get into the system, the more holes can be exploited. The article did mention that they seem to be using a sophisticated root kit, after initial entry.
> as I read it was mainly machines with cpanel that were infected.
However many infected had neither cpanel or the other common app. Likely, there are multiple vectors. This seems
Re: (Score:2)
And this is why I usually announce a version with a well known exploit and keep the firewall trained to alert me of exploits targeting that version.
No better way to tip you off to a targeted attack.
Re: (Score:2)
Re: (Score:2)
It can very easily backfire, of course. Takes some serious titanium balls to pull it off. :)
Re: (Score:2)
Keyword here is 'if'.
Most attackers will just run every exploit against a service in the hope that one will stick.
Re: (Score:3)
It's going to be useful if someone is trying a targeted attack directly on your server. That way they know what version you are running, and can go to the correct source code trying to find a vulnerability, and not waste time on newer versions, or older versions, or patched versions, or whatever.
Not really. Even targeted attacks are begun by an automated probe that just tries everything and sees what sticks.
It's a foolish thing to care about. The headers shouldn't be there, and if they are, it shouldn't matter. It takes literally seconds or minutes to ferret out what the tool is, and often the site says so right in a page somewhere.
Caring about it is for fools or people who want to lead those fools away from their money.
Re: (Score:2, Informative)
So how do you do that in Apache?
You're looking for ServerTokens and/or ServerSignature.
But as the comment says:
# Changing the following options will not really affect the security of the
# server, but might make attacks slightly more difficult in some cases.
Re: (Score:2)
This is true but you are assuming that removing this information from where it is now presents a significant barrier to such a database. I find this unlikely with the ease of both os level and application level fingerprinting and the availability of throwaway botnetted machines.
This protects best against the very specific MO of someone with a few exploits looking for a wide range of servers to exploit. This seems more targeted. If they didn't have the information needed on a desired target, they could likel
Re:I have a stupid question. (Score:4)
It's all about advertising, to show just how many people use their webserver.
Why? (Score:5, Interesting)
Re:Why? (Score:4, Funny)
Are you afraid of little infected web site? Something wrong with your browser?
Re:Why? (Score:5, Insightful)
Re:Why? (Score:4, Funny)
Find out what they're experts in, become a complete idiot in that field and start pestering them with requests for help.
Keeps my dad away. Though I now have to pay for repairs when my car breaks down.
There is something wrong with EVERY browser (Score:2)
There are numerous security flaws in all the major browsers. Vulnerabilities are getting fixed all the time; just look at the change log of Firefox or Chrome over the last few releases, for example. If you think you're magically virus-proof because you're running your pet OSS software, you might consider the list of popular OSS web servers in the title of this discussion.
Re:There is something wrong with EVERY browser (Score:4, Insightful)
In my experience most of the major browser exploits attack vulnerable plugins (flash, java, acrobat/pdf viewer, etc) or abuse scripting.
If you restrict or disable said plugins and javascript then I'd say you're pretty darn safe.
Granted, most "web 2.0" websites work like shit without javascript enabled but some stuff still works. For the more sane of us there are things like NoScript.
It's kind of hard for plain text and images to do bad things though I suppose it's been done before.
Re:There is something wrong with EVERY browser (Score:5, Interesting)
From Debian 7 release notes:
"Therefore, browsers built upon the webkit, qtwebkit and khtml engines are included in Wheezy, but not covered by security support. These browsers should not be used against untrusted websites. For general web browser use we recommend browsers building on the Mozilla xulrunner engine (Iceweasel and Iceape) or Chromium."
-- http://www.debian.org/releases/stable/amd64/release-notes/ch-information.en.html#browser-security [debian.org]
Re:There is something wrong with EVERY browser (Score:4, Interesting)
They attack plugins because flash/java/acrobat are still installed on over 90% of potential targets, whereas the browser market is now diversified...
Re:There is something wrong with EVERY browser (Score:4, Informative)
It's kind of hard for plain text and images to do bad things though I suppose it's been done before.
There have been vulnerabilities in PNG and JPG image format handlers in the past, so yes, there has definitely been the potential to have images do bad things. (Arguably none would be as bad as using some of the ones relating to goatse, but that's a different kind of problem.) If you hear of problems in fundamental media type handlers, for goodness sake make sure you're up to date with your security patches!
I don't know if there were any exploits of those problems in the wild though.
Re: (Score:1)
I have several free web browsers on my laptop, but I generally do not look at web sites from my own machine, aside from a few sites operated for or by the GNU Project, FSF or me. I fetch web pages from other sites by sending mail to a program (see git://git.gnu.org/womb/hacks.git) that fetches them, much like wget, and then mails them back to me. Then I look at them using a web browser, unless it is easy to see the text in the HTML page directly.
I think this is the key.
Get your friends' computers infected instead!
Re: (Score:3)
There's a small number of infected sites. That clearly indicates that this is likely a case of digital burglary rather than the much lower bar of something like a viral infection. Otherwise we would be talking about thousands of sites or half the Internet.
Your screed would be more relevant if not for the fact that there are various fairly common workarounds employed on the various browsers to mitigate just this kind of nonsense.
A little paranoia goes a long way. That's far more useful than the sort of bliss
Re:Why? (Score:4, Interesting)
How exactly does your browser recognize the difference between a normal page and the exact same page delivered from the exact same server at perhaps a microsecond delay?
This backdoor may simply be passing on POSTs with passwords (a webserver receives these unencrypted, you know) to another server without altering anything on the page. The only one who'd notice would be a webserver admin that happens to monitor outgoing traffic.
Re: (Score:3)
Also, the sites they've found are probably not infected anymore, since presumably they've been notified and resolved the problem.
Re: (Score:2)
It could lure you into a sense of false security, letting you think you are safe by avoiding them, when really you don't know that. Other sites are probably infected too.
methinks the whole interntets build upon a false sense of false security. the OP is right, there is no reason not to disclose the list.
Also, the sites they've found are probably not infected anymore, since presumably they've been notified and resolved the problem.
this is a bold assumption, and a clear indication of a false sense of security :-)
(besides in contradiction with your previous statement)
Re: (Score:2)
Re: (Score:1)
sorry. there's no contradiction, really, i meant that warning against trusting *any* site (an advice i endorse) would be incompatible with advocating for trusting *some* sites because they would fix the issue (the same sites that got pwned in the first place), but i see now that you weren't implying that at all. my bad.
Re: (Score:2)
Re: (Score:2)
1. slashdot.org ....
2.
Too late!
Re:Why? (Score:4, Interesting)
Why isn't there a list of infected sites? Avoiding them would seem to be a priority.
Here is how to make sure you are not one of the infected sites: Compile and run this:
http://www.welivesecurity.com/wp-content/uploads/2013/04/dump_cdorked_config.c [welivesecurity.com]
If you don't want to vet that, you can get a first-aproximation with "ipcs", just look for the Apache PID, which you can get from "ps aux | grep apache2".
Re: (Score:2)
Just because you see a shared memory segment used by apache doesn't mean that you're infected. Apache sometimes legitimately uses shared memory segments. See, for example: http://blog.nominet.org.uk/tech/2008/03/26/apache-shared-memory/ [nominet.org.uk]
Thank you. That is an interesting use case, one that I had never encountered. Obviously, if Apache has been configured to share memory across processes then seeing it do so is not clear sign of infection. However, if Apache has not been explicitly configured to do so, then seeing Apache sharing memory with another process is a real red flag.
Your linked blog is great, there are quite a few gems in there. Thanks!
Re: (Score:2)
It's a nice filter. If apache isn't using shared memory, you are ok. If it is, well, research further.
Name the 50 sites (Score:5, Insightful)
The actual quote is, "50 are ranked in Alexa’s top 100,000 most popular websites." Quite different than the summary but would still be interesting to know.
Re: (Score:2)
That Alexa put some list together and these sites happen to be on the list of affected sites - is immaterial.
It's material because it means that you had a nontrivial chance of actually running into them.
Also, releasing that information is less useful than you think, because presumably those sites have been notified and resolved the problem. The real security worry is the sites that are infected but we don't know. They are definitely out there.
Re: (Score:2)
Well that really depends if the affected sites are #1 to #50 or #99950 to #99999 now doesn't it ?
checksums (Score:2)
The hack resides in memory. (Score:2)
Re: (Score:3)
I think they said there is a modified httpd although. It should be enough to raise suspicion.
https://www.net-security.org/secworld.php?id=14836 [net-security.org]
And they still don't know the initial vector (Score:4, Insightful)
We also don’t have enough information to pinpoint how those servers are initially being hacked, but we are thinking through SSHD-based brute force attacks.
So does this mean I need to remove sshd? Doubtful. More likely the initial vector is social engineering or weak passwords (social stupidity). That makes this whole infection uninteresting ... it's just an app from the web server perspective. OK, so it can break into your browser with a zero-day. Fix the browser.
Re:And they still don't know the initial vector (Score:4, Informative)
So does this mean I need to remove sshd?
No, it means you need a more complicated password.
And it seems to be just a guess, they probably came to 'sshd' by following a line of reasoning starting with the only thing they could think of that all the hacked servers have in common.
Re: (Score:3, Informative)
Re:And they still don't know the initial vector (Score:5, Informative)
Worried about exposed sshd? Install pam-abl and watch the brute force attackers waste their time. With my config, three failures from any IP address in an hour (or 6 per day) and that IP is locked out for a week through PAM. They can still try, of course, but even if they somehow guess the correct password, it must be in their first three guesses each week.
There's no indication to the attacker that pam-abl is there, and there's very little chance of a DOS attack against legitimate logins.
Oh, and you've denied root logins from the internet, haven't you?
Warning: Source tarball, but if I debian-ized it, then anyone can.
Re: (Score:3, Informative)
There are quite a number of ways to harden access
1. pam-abl (as noted above)
2. denyhosts
3. VPN (openvpn works for me)
4. Hosting ISP firewall
Also as noted above, do not permit direct remote root access. Doing anything less is just advertising yourself as a platform for malware.
The first three are quite easy to set up. There is really no excuse for not setting up a least a minimum level of security on your system. That plus careful use of mod_security, and you've done quite a bit towards thwarting the cas
Re: (Score:2)
Here's what I do:
1. Have sshd listen on a non-standard port (ie NOT 22)
2. apt-get install fail2ban - lock the
3. Use pam_listfile to whitelist what accounts can log in with SSH
4. Explicitly disable SSH root access in sshd_config
pam-abl doesn't solve this (Score:3)
Re: (Score:3)
Worried about exposed sshd? Install pam-abl and watch the brute force attackers waste their time.
Thanks for the link to pam-abl. That was the first I'd heard of it. Neat module.
Personally, I've always gone with
RSAAuthentication yes
PubkeyAuthentication yes
PasswordAuthentication no
and I sleep well at night. Although I have to admit, I sure see an assload of this type of crap in the logs:
May 9 11:12:38 imap sshd[15366]: reverse mapping checking getaddrinfo for ras-185-151.wntpr.net [196.12.185.151] failed - POSSIBLE BREAK-IN ATTEMPT!
May 9 11:12:38 imap sshd[15366]: Invalid user matt4 from 196.12.185.151
Re: (Score:2)
The simple solution there is to move SSH off of the default port (of 22) and to some other port in the 1-1024 range. You'll end up with a lot less crap in the log files as a result.
Which makes it easier to see the real threats because they aren't camouflaged by hundreds of other errors in the logs.
Re: (Score:2)
Yeah, you could do that, or you could change to a non-standard port. In my ten years of running sshd on a non-standard port on several public servers I've been hit by exactly 0 (zero) probes on that port.
Obscurity for the security win!
Re: (Score:2)
So does this mean I need to remove sshd?
I got an e-mail in my spam folder last week so I pulled all the hard disks.
Unlike you other lameoids, my server aints gettin hacked.
Re: (Score:2)
No, but if you can you should disable pasword authentication.
Re: (Score:2)
Any public side SSH service where you are only using SSH for administration of the machine should be:
1) Disallowing password-based authentication. Use only SSH key pairs instead for authentication. Now the attacker also needs to steal your private SSH key (and possibly find out the password as well). You've just made it a lot more difficult for them.
2) Moved to an alternate port. This doesn't make you immune to attacks, but it does mean that you'll see less
Re: (Score:2)
...and the server wasn't using any of the various forms of brute force attack countermeasures.
These come prepackaged now but you could easily craft one yourself out of basic Unix tools. Did that very thing before discovering fail-to-ban.
A little paranoia goes a long way.
Re:And they still don't know the initial vector (Score:4, Informative)
And if not fail2ban, a good first step is updating the firewall rules to have a rate limiter on sshd. Mine allows only 2 attempts to connect a minute.
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -m recent --set
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -m recent --update --seconds 60 --hitcount 2 -j DROP
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
Re: (Score:2, Informative)
Also don't use user/password ssh logins. RSA authentication only.
Re: (Score:2)
Has this never resulted in denial of service?
Fix (Score:5, Funny)
You can download a fix here [iis.net].
Re: (Score:1, Funny)
Yes, indeed. Why suffer from this minor malware when you could have all the best ones infecting you? Lightweights!
Re: (Score:2)
Cpanel? (Score:1)
is this for cpanel or apache?
Re: (Score:1)
The intruder is backdooring the binaries. It has nothing to do with cPanel. Not to mention cPanel runs easyapache, but if it became an intended target, im sure it could be infected just the same...
proves this is all a farce (Score:2)
By "backdooring the binaries", I can change the operating system or any software of any system. So a system gets rooted, and bad wares can be installed. Bear shits in the woods, story at 11.
Re: (Score:1)
Re: (Score:3)
is this for cpanel or apache?
TFA [net-security.org]
"We still don’t know for sure how this malicious software was deployed on the web servers," the researchers admit. "We believe the infection vector is not unique. It cannot be attributed solely to installations of cPanel because only a fraction of the infected servers are using this management software. One thing is clear, this malware does not propagate by itself and it does not exploit a vulnerability in a specific software."
Curious (Score:1)
Only 2 weeks back, when this was reported everyone blamed cPanel. Now that exposures in NGNIX and LighHTTPD have been found, comments (guesses) as to the attack method are proposed, but very few realistic ideas. TFA seems to indicate that it's more pervasive than many of the people commenting want to believe. I'm just waiting for them to find IIS infected as well.
Not an apache/nginx/lightttpd vulnerability (Score:3, Informative)
Those servers somewhat (i.e. a vulnerable web app, weak ssh passwords, local privilege escalation on a shell got in in some way, or a combo of all of those) got rooted, and instead of modifying web pages (easier, but also easier to detect and fix), replacing the entire web server (easier to detect or to roll back) or changed the configuration of i.e. mod_rewrite modules (that with a configuration manager could had been detected/roll back to the original one). got some new modules replaced/added, modules that in particular had that functionality.
Is nothing particulary new in this, more than the malware authors not being just script kiddies and actually did some serious programming for it. Somewhat I hope that they give back to the community releasing the source, not the malware backdoor itself, but with a modified, non malware version with an useful use (i.e. something that dynamically blacklists IPs/useragents/languages for actions, receiving the input from another kind of system, like a honeywords [slashdot.org] service) if not available yet.
Those idiots at Microsoft (Score:1)
If they'd used Linux instead, this wouldn't have happened.
Re: (Score:1)
Re: (Score:1)
That girl's standing over there listening and you're telling him about our back doors?
Clearly, that girl is interested in back doors because he has a package at his front door.
screw it (Score:5, Funny)
Re: (Score:1)
I don't think you're familiar with the stereotypes. You'd only need to secure your ass if you were going to OS X.
Re:and this is why.... (Score:5, Funny)
FreeBSD runs the same software stack, so it would make little difference.
That's why our organization uses a custom server software written in 68K assembly running on MacOS 7.6.1 on a cluster of Quadra 610s.
Re: (Score:2)
That's why our organization uses a custom server software written in 68K assembly running on MacOS 7.6.1 on a cluster of Quadra 610s.
Well played sir, well played indeed.
Re: (Score:1)
610s? I can get 40 605s in a rack that fits 10 or so 610s.
Which 10G AAUI modules are you using for your AppleTalk SAN?
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)