CensorNet

FAQ Section 4 - Blackist Update (BLUD) Questions

Questions

Q1.1 How do I sign-up for a Blacklist Update (BLUD) Subscription?
Q4.2 I have been sent my username and password - now what do I do?
Q4.3 I have entered my username and password into the setup program. What happens next?
Q4.4 How do I start/stop the BLUD process manually?
Q4.5 How do I stop the BLUD process forking into the background?
Q4.6 I get the error "407 Authorization Failed" in the BLUD log file - what does this mean?
Q4.7 I get the error "500 Download Error" in the BLUD log file - what does this mean??
Q4.8 What does THOROUGH MODE mean?
Q4.9 Yesterday it said I had 500,000 entries in the blacklist database but this morning I have none - help!
Q4.10 Why does it take so long to process a BLUD update packet?
Q4.11 I get the error "Testing the integrity of gzip file: Bad File Descriptor" - what does this mean?
Q4.12 I get the error "500 EOF instead of response status line" - what does this mean?
Q4.13 Can I use CensorNet without subscribing to BLUD?
Q4.14 How often does the blacklist get updated?
Q4.15 Can I export the blacklist from CensorNet?
Q4.16 I get a timeout error every time BLUD runs, so my list isn't being updated. How can I fix this?
Q4.17 In CN3.3 you use transactions by default to update the BLUD list. Why do you offer the option to turn it off?

Q4.1 How do I sign-up for a Blacklist Update (BLUD) Subscription?

A4.1 Visit our online store or if you cannot pay by credit card please contact a member of our sales team for alternative methods of payment. Your username and password is normally issued within 24-48 hours of receipt of payment.

Q4.2 I have been sent my username and password - now what do I do?

A4.2 Run the setup program and choose "Blacklist Update Configuration". Enter your username and password in the box provided. Optionally choose the "Advanced" to set additional options such as the purge interval and report verbosity.

Q4.3 I have entered my username and password into the setup program. What happens next?

A4.3 On the home page of the CensorNet Web Interface a new panel will be displayed showing the Blacklist Status. It is perfectly normal for it to say "FAILED" after you have just entered your username and password. The BLUD process will start at the time you have specified (default 01:30). Again, the new panel on the home page won't change until the BLUD process has finished (default 05:00) and then the panel will show the latest statistics including the number of new entries in the blacklist database.

Q4.4 How do I start/stop the BLUD process manually?

A4.4 To start the BLUD process, login as root and type:

/usr/bin/blud/blud.pl

You will be returned to the prompt as the BLUD script forks into the background (see A4.5 for details on how to stop it forking). It is sometimes useful to use the command:

tail -f /usr/bin/blud/logs/blud.log

to view the progress of the BLUD process and check for any errors.

To stop the BLUD process, login as root and type:

killall -INT blud.pl

It is important that you don't use kill -9. You must send the script the INT signal so that it knows to shutdown gracefully. Stopping the process any other way is not recommended.

Q4.5 How do I stop the BLUD process forking into the background?

A4.5 Use your favourite editor to open the file /usr/bin/blud/blud.pl and search for the variable $FORK_ME_BABY (as in Fork me, baby!). Change it's value from 1 to 0 (zero) so that it looks like this:

$FORK_ME_BABY = 0;

Next time you run the process the output will be sent to STDOUT. Remember to change it back after you have finished.

Q4.6 I get the error "407 Authorization Failed" in the BLUD log file - what does this mean?

A4.6 This means the username and password you have entered into the setup program are invalid (remember they are case sensitive) or your subscription has expired or been suspended. It may of course be a problem with our server so in any instance please e-mail support@censornet.com.

Q4.7 I get the error "500 Download Error" in the BLUD log file - what does this mean?

A4.7 Ensure Use for All Requests is selected in the Web Cache Configuration section of the setup program.

Q4.8 What does THOROUGH MODE mean?

A4.8 Thorough Mode is a hang over from old versions of CensorNet. When in thorough mode, the BLUD process will "visit" every site in the blacklist update packet and check if it is blocked by the current filtering rules you have configured, and if it is, will not add it to the database. This feature will become extinct in future versions of CensorNet.

Q4.9 Yesterday it said I had 500,000 entries in the blacklist database but this morning I have none - help!

A4.9 You need to apply the BLUD Patch which fixes a bug in the purge interval. This has been fixed in 3.2r3 and later.

Q4.10 Why does it take so long to process a BLUD update packet?

A4.10 The BLUD process checks every site in the blacklist update packet to see if it is still "live". If the site is still running, it's timestamp is updated with the current date/time. This process is repeated for every site in the update packet (there may be over a million sites).

Why does it do this?

Porn sites in particular come and go, often they are only live for a few days. The database would get clogged up with dead sites if the BLUD process did not check the validity of every URL in its update packet. By updating the timestamp of when the site was last live, the database can be purged every so often so that old and broken sites drop out of the list.

Q4.11 I get the error "Testing the integrity of gzip file: Bad File Descriptor" - what does this mean?

A4.11 This is usually due to a corrupt download. Try running the BLUD process manually (see A4.4) or change the time that it runs to avoid conflicts with other system processes. It is recommended that you run BLUD after 1am.

Q4.12 I get the error "500 EOF instead of response status line" - what does this mean?

A4.12 See A4.11.

Q4.13 Can I use CensorNet without subscribing to BLUD?

A4.13 Of course! CensorNet does come with a comprehensive blacklist so you can get up and running quickly and safely. We recommend you subscribe to BLUD to keep you up-to-date as a stale blacklist database isn't much use at all! See A4.1 for details on how to subscribe.

Q4.14 How often does the blacklist get updated?

A4.14 We update our master blacklist on a daily basis. You may not necessarily download a new update every day - it depends on how long it takes your CensorNet server to process its current packet.

Q4.15 Can I export the blacklist from CensorNet?

A4.15 Not easily. You would need to make your own script to export the sites from the database.

Q4.16 I get a timeout error every time BLUD runs, so my list isn't being updated. How can I fix this?

A4.16 This normally happens to people who have a parent proxy.

By default, the BLUD update doesn't look for parent proxies, and doesn't use them. If your upstream provider has blocked all web access, except when it goes through the parent proxy, you can suffer this problem.

You need to log into the CensorNet as root and type the following commands :-

cd /usr/bin/blud
vi blud.pl

The file is now in front of you. Type

/USE_PROXY

This should take you to a line showing that its value is currently 0.

Use the k key to move the cursor to the right, until its on the 0 and type

r1

Now save the changed file by typing

:wq

This command should appear at the bottom of the screen, and after you've typed the RETURN key, it should tell you how many lines and characters its written.

That's it, job done.

Q4.17 In CN3.3 you use transactions by default to update the BLUD list. Why do you offer the option to turn it off?

A4.17 Some older hardware simply can't cope with transactions, and bails out part-way through a packet update. So we provided the option for those machines to be able to disable it. Despite that, other changes we made to the BLUD script mean that this older hardware can still process a full packet during a night's running, instead of taking several days.