Today some dirty spammer created about 25 fake accounts on the site, no doubt in an attempt to post links to his Viagra sites. Since not one single one of these accounts was ever able to make any posts I thought I would share my simple spam-fighting techniques.
One of the best spam-fighting techniques is built right into Drupal - eMail validation. Requiring email validation is what foils many spammers. In order to make posts they must validate their account by providing a valid email address, then clicking the validation link sent to that email address. Since Drupal does not allow duplicate email address, it takes a lot of work for the spammer to go create new email accounts. Most spammers don't bother and move on.
The other great way to avoid spammers in Drupal is using the Bad Behavior module. Bad Behavior stops most spammers, bots and other automated systems in their tracks. Note that if you use the LM_PayPal module for processing incoming PayPal payments you will either have to disable Bad Behavior, or make sure that all the PayPal servers are white-listed.
The last and most low-tech method for spotting spam and fake accounts is to watch your Drupal logs and bounced emails. If you see 50 new accounts created over a period of 1 minute, all with names like "Sue", "Ralph", "Sammy", "Paula", etc, you can be pretty sure they are fake. Watching bounced emails will also tip you off to fake accounts as most spammer's email address are either fake, or over-quota.
Using these very simple techniques have helped to keep GrownUpGeek.com 99.98% spam-free for 18 months.
Labels: Drupal, spam, The Site
¶ 9:46 AM 0 Comments
Anybody that knows anything about SEO will tell you over and over to avoid duplicate content. Usually this means don't copy and paste other people's work on your website, or don't buy turnkey sites that display feeds for article submissions, etc.
But if you use Drupal there is another way you can get hit with a duplicate content penalty without even knowing it by improper use of the dreaded "/" - That's right, the / (backslash) character. As it turns out, if you use Drupal it treats www.yoursite.com/a-page and www.yoursite.com/a-page/ as different pages. This means that if a search-engine bot comes to your site via a link with trailing "/" it could potentially index several duplicate pages or at worst a duplicate of your entire site.
I had never heard of this potential issue with Drupal so big kudos to my new friend Alex of pitumbo.com who I met at the April WEMUG meeting. Alex pointed me to this article at blamcast.net that explains it better than I ever could (please take a minute to read it and give it a Digg).
Basically the trick here is to use a 301-redirect to remove the trailing slash at the end of all your URL's. The code to put in your .HTACCESS according to Blamcast.net would look something like this:
#remove trailing slashes
RewriteCond %{HTTP_HOST} ^(www.)?yourdomain\.com$ [NC]
RewriteRule ^(.+)/$ http://%{HTTP_HOST}/$1 [R=301,L]
I quickly tossed this code into my .HTACCESS file and loaded a few test pages and to my surprise it did not work. I think the WWW -> non-WWW redirect I'm using to remove WWW from all my URLs was affecting it so after some tinkering I ended up with this version which seems to work:#remove trailing slashes
RewriteCond %{HTTP_HOST} !^\.grownupgeek\.com$ [NC]
RewriteRule ^(.+)/$ http://%{HTTP_HOST}/$1 [R=301,L]
If you're using Drupal and are looking to squeeze every bit of SEO from it, I recommend this simple change. Thanks to blamcast.net for putting it out there.
Google is very picky about duplicate pages and may penalize your site if it finds too much (or any?) duplicate content. To help squeeze a little-bit more SEO friendliness out of Drupal you can add a few entries to your ROBOTS.TXT to avoid duplicate pages from being indexed and keep the search-bots away from pages that they don't really need to be in.
If you are using Drupal 5 or older, try adding these entries to your ROBOTS.TXT
User-agent: *
Disallow: /admin
Disallow: /aggregator
Disallow: /tracker
Disallow: /node/add
Disallow: /user
Disallow: /files
Disallow: /search
Disallow: /book/print
Disallow: /filter
Disallow: /filter/tips
Disallow: /comment
Disallow: /comment/reply
Disallow: /spam
Disallow: /spam/report
Disallow: /spam/report/comment
I was browsing through the Drupal Support Forum and this post about the Drupal WSOD (White Screen of Death) caught my eye. I could feel the OP's pain as I read part of his post:
"I physically moved the Tagadelic subdirectory out of the modules directory. This hasn't had any effect: just whiteness, whiteness everywhere. Argh."
At first, it was the "WSOD" (White Screen of Death) reference that grabbed my attention. Back in September 2006 GrownUpGeek.com suffered from a particular nasty case of the Drupal WSOD, which kept the site down for about two days and resulted in the loss of several posts and all user accounts. It was at this point in time that I learned the importance of backing-up your Drupal database often.
The second thing that caught my eye about the post was how quickly the OP's problem was solved - about 3 hours. Support for your Drupal install can be hit or miss at the forums. In some cases your pleas for help and begging for answers will fall on deaf ears and blind, cold, heartless eyes. In other cases such as this WSOD post, and mine from last week, the Drupal Gods save the day.
Labels: Drupal
¶ 8:11 PM 0 Comments
We recently installed the Drupal Userpoints module at the site to track posts, rate users, etc. I was running into issues and decided to upgrade to the new Userpoints 2.0.
After upgrading/installing the newer version I continued having trouble so late Saturday evening made a post in the support-forum at Drupal. To my surprise within hours I had a response from Khalid Baheyeldin, the author of the Userpoints module and co-founder of 2Bits.com.
I Responded to Khalid's post Sunday morning and was again surprised to see multiple responses almost immediately after my posts. Khalid then offered to access my system and look into the problem directly. Normally I would never even entertain the idea of giving a complete stranger the passwords to the site, but I have seen (and used) many of Khalid's Drupal modules and 2Bits.com was already on my list for any future Drupal support (like that upgrade to V5 that I'm so afraid to do myself). I emailed the key's to the GrownUp Geek kingdom to Khalid and held my breath.
Long story short: A few minutes later Khalid sent me an email indicating the problem was resolved. A few more very fast emails back and forth, and the Userpoints module was working as advertised.
I have been disappointed with the level of Drupal support in the forums in the past, but in this case not only did I receive a response to my problem, but I was also offered and received personalized support - all on a Sunday morning! Kudos to Khlid, 2Bits.com and Drupal on a job well done!
Naturally, I do not expect this level of free support for all problems (nor should you), but my experience today proves that there are some great Drupal developers out there that not only know what they are doing, but aren't "too good" to help us n00bs.
Thank you Khalid and 2Bits !
Labels: Drupal, Site Development, The Site
¶ 12:43 PM 0 Comments
How to make Drupal faster seems to be my mantra lately. In my previous post, "Drupal is still faster" I bragged about how disabling User-Pictures (avatars) reduced CPU overhead dramatically, and greatly improved the response time of the site. The only problem is that as GrownUpGeek.com becomes more of a social-site, those user avatars are important!
I posted my user-pictures CPU usage problem at the Drupal Support Forum, and as expected have gotten zero responses. That was expected, seeing as how Drupal is free, and nobody is obligated to help or answer - leaving me to figure out the problem all on my own. What I have found after hours of Googling is that the GD Library is responsible for the excessive CPU usage while rendering the avatars. This makes sense but it does not explain why only some avatars cause problems.
Due to public demand I have re-activated the user-pictures but have chosen to display only certain user's avatars (like mine, moderators, etc) in the forums. I accomplished this by creating a new, hidden checkbox field in all members user profiles with the Profile module that I could then use to choose which members I want to have avatars displayed. I then added this PHP code to my node-forum.tpl file (I'm using FlatForum):profile_load_profile($curr_user);if ($picture && $curr_user->profile_useImage) {print $picture;}
This queries the "profile_useImage" field that I created to determine if the avatar should be displayed or not. I only did this for the forums because that is where most traffic is, but it could also be used in nodes/blog template files as well. I found how to do this in the Drupal Forums, so even though I dont always get answers, it is still a great resource.
In addition to only displaying selected member avatars I have also further reduced the size limits on avatars to only 50x50px and 10kb in size. I will try these methods for a few days to see how the CPU usage looks. If CPU usage is still too high, and if I still cant get any answers to the GD Library CPU-hogging problem I will probably just remove all avatars from the forum areas altogether. I can do this by removing the "Profile_load_profile" code (above) from the node-forum.tpl.php file. This will prevent user-pictures from displaying in the forums, while leaving them in other areas of the site.
A quick follow-up to my post from the other day, Making Drupal faster, which was a followup to my post from a month or so back, Drupal Optimization:
After disabling user-avatars at Grown Up Geek, CPU usage has plummeted from daily averages of 35% - 55% down to an amazing 11%-13%. I can say with confidence that this single, simple change has made the greatest impact on site performance since moving from a shared host to our dedicated server.
If your Drupal site is running slow and you have user-avatars enabled, try disabling this feature to see if you get the same astounding results that I did.
Labels: Drupal, Site Development, The Site
¶ 8:59 PM 0 Comments
Now that GrownUpGeek.com is regularly getting over 10,000 unique visitors per day (about 40,000 page views), and quickly closing-in on 15,000 visitors per day, I've noticed that performance isn't as great as I think it should be. Page loads for most pages are fine, but creating new forum threads or posting comments can sometimes take 15-20 seconds, and page loads on pages with many comments can sometimes take several seconds.
I figured I should have a look at the CPU usage to see where the bottlenecks might be. A quick check of CPU Usage in cPanel (I'm still pretty clueless about Unix so I have to rely on cPanel WHM) showed me that on most days the top CPU-sucker was the HTTPD (web server) process chewing on one or two .JPG files that turned out to be member avatars. Most of these avatars were from members that haven't been active at the site in months, but had posts on pages or forum threads that are now getting a lot of traffic. Since these members aren't around anymore, I deleted the particular avatar files that were sucking my CPU. Immediately after removing these files I noticed a huge improvement in page load times, and overall performance of the site. I'm not sure why something as simple as a .JPG avatar would make Drupal run so slow, but I think it has something to do with how Drupal renders the images, which is CPU intensive. In an attempt to help prevent this problem from getting worse, I reduced the allowable avatar size down to 70x70 pixels, and reduced the allowable file size to 35kb.
For the last few days since I've been watching the CPU usage and removing these old CPU-hog avatars the site has been running much better..
*Update: My CPU usage logs was still showing high usage from more avatar files. I have disabled avatars (Drupal admin menu, users, "picture support:Disabled") - after disabling user-picture support site performance increased noticeably. Moral: Drupal user avatars slow down the entire site!
For a few more ways to make Drupal faster, see my post from a few months ago here:
Speed-Up Drupal
Drupal is a great CMS but it's a bit top-heavy. I don't know how it compares to other CMS's but from my experience it can really push your CPU and memory usage. When we moved to our dedicated server I thought that the almost daily site-crashes would be a thing of the past, but within weeks of transferring our site as soon as traffic started to increase we ran into trouble.
These are some of the things I learned to minimize CPU and memory usage:
- Activate the Drupal cache: The built-in Drupal cache displays cached pages to anonymous users, saving the overhead of database queries to build every page. Members that are signed-in get uncached pages
- Disable un-used modules. Don't just disable them, but remove them from the modules directory completely. Even when the modules are sitting there deactivated, they still consume memory
- Disable the Statistics module: The Drupal statistics module is great. It tracks every pageview and every referrer and is a great way to see what is happening at the site. The only problem is that it tracks every pageview and every referrer - creating a significant load on the CPU - causing a 30%-40% increase in CPU usage during peak traffic. I still turn the statistics module on from time to time, but it's usually off.
- Disable (or don't use) too many filters: I disabled the WordFilter and Smiley's filter. We have three moderators so the WordFilter isn't necessary anymore and we don't need the Smiley filter because TinyMCE has it's own smileys
- Clear the Sessions Table: Once a week I put the site into maintenance-mode and empty the sessions table. If you get a lot of visitors this table can grow quite large. After clearing the sessions table you and everyone else will get logged-out and will be forced to log-in the next time you access the site. In case you forget, the url to log yourself back in is http://yoursite.com/user if you have SE Friendly URLs activated, http://yoursite.com/?q=user if you don't
- Clear the Cache: When I clear the sessions table, I also clear the cache. Before doing this, I disable the cache
- Optimize Tables: After clearing the cache and sessions tables I optimize all of the database tables
- Upgrade your RAM: I was forced to upgrade from 1GB to 2GB of server RAM. Prior to the upgrade my memory usage was floating around 75%-90%, and would occasionally hit 100%; swap usage was also very high. After the upgrade to 2GB, memory usage floats around 38%-45% and rarely tops 50%; swap usage is usually now 0%
Labels: Drupal
¶ 9:32 AM 3 Comments
We recently had the unpleasant task of having to 'ban' a member from the website. Without getting into the drama and details, we'll just say this particular member was less than trustworthy or honest - and in a community like GuG, both of these qualities are necessary.
How to ban a user with Drupal:
With Drupal there are a few different ways to ban a member from the site. But no single one is 100% effective. We used a combination of methods to carry-out the banishment.
BLOCK:
Drupal as the ability to block a member by login name. To block a user using the Drupal BLOCK method just edit the user properties and click the "Blocked" radio-button under the status section. The only problem with this method is that it will prevent that particular user-name from logging in, but it will not keep that person from creating a new member name.
TROLL:
The TROLL module for Drupal is invaluable. Not only does it allow you to do a second level of blocking via IP, it tracks all the IP address of all your visitors. You can then easily do a query of a particular IP to see what members have used it, or you can query a member to see what IP's they have used. The Troll module also allows you to block IP's. All IP's entered into the Troll "block" list are redirected to a customizable "you've been blocked" page. Each time a blocked IP tries to access your site, a message is listed in the logs so you can see how often the user/IP tries to get back to your site.
HTACCESS:
The next level of blocking can be done at the HTACCESS level. The Troll module still let's the user "hit" the site and database. Blocking via HTACCESS will not even allow them to get to the site at all. To block or ban with HTACCESS, you need the IP which is easily obtainable if you're using the Drupal Troll module.
Sample of using HTACCESS to block an IP:
order allow,deny
deny from xx.xxx.xx.xxx
deny from xx.xx.xx.xx
allow from all
With HTACCESS you can also block via referrer, domain, ip-range, and more.
Server Level:
If you run your own server and you want to pull out the big-guns you can block the IP at the server. This will prevent access to the website, email, or anything else on the server. You can easily do this with IPTABLES in Apache using this commend:
iptables -I INPUT -s xxx.xxx.xxx.xxx -j DROP
We havn't had to resort to this method yet, but it's there if we need it.
Blocking by IP isn't 100% effective because IP's change - particularly with dial-up users. But if you use the Drupal Troll module and the member has been around for a while, it will give a complete list of all the IP's that he/she has been using. By using all the methods above you can create a pretty effective ban.
Labels: Drupal
¶ 7:43 PM 0 Comments