Re: [Amps] Choices for valve linear project 1st timer!

[David Kirkby <david.kirkby@onetel.net>]

R.Measures wrote:

> On Feb 1, 2005, at 4:26 AM, Simon Steed wrote:
>
> Anybody is welcome to the entire contents of my Web site because that 
> was my intent.
>
> Richard L. Measures, AG6K, 805.386.3734.  www.somis.org 


Rich,

Same here - if I did not want anyone getting it I would not put it on a 
web site.

BUT it can become a BIG problem if lots of people start downloading HUGE 
chunks from your site, which is what has been happening with me at

http://www.g8wrb.org/

and perhaps Ian too.  These people are not downloading what they *need*  
but just everything they possibly can. Looking at last months figures, I 
see 3.83 GB was downloaded and viewed, but more than this (5.22GB) was 
not viewed. Some of that 5.22GB is taken by idiots just downloading the 
whole site.

I run my own web server  (by that I mean I use an ISP for connectivity, 
but I don't use their web server). I use my own server (to which I 
dedicate a machine for security reasons). That gives me a lot of 
control, and I can see what happens.

Ian's  ISP *might* give him access to the log file. Each domain hosted 
can have its own log file, so there is no good reason they will not give 
you access to that. They can however set how much data is logged, and if 
they log the default amount on the Apache web server, it is quite small. 
ISP's will probably do that, as it saves them having big log files. So 
if you get the logs, you might ask them to log more.

I'll post a bit of a log below. You can see

1) One IP address 68.0.150.42 is using 'HTTrack' That is designed for 
copying web sites. He/she is not downloading material they want.

2) He has downloaded tube data sheets, data on a particular 5.5 kW UPS, 
material on cooling a Sun SPARCstation 20, material on Yagis.  Now with 
the best will in the world, it is most unlikely anyone is likely to own 
a SunSPARCstation 20, this particular HP 5.5kW UPS, need these range of 
tube data sheets ... and so on.

I've no objections to anyone taking a few of those items, but it does 
get annoying when your site is being slowed (and/or you pay more) 
because some individuals are taking everything they can get.

3) Assuming your ISP uses Apache (used more than all other web servers 
added together), there is the 'mod_bandwidth' plugin. I've not used it, 
but it can be configured to slow the connection to those taking huge 
amounts. That might be what Ian could do with, but unless he runs his 
own server (or pays someone to professional administer one), he is 
unlikely to be able to configure this.

4) If you have a lot of useful data, you might try what I have done, 
which is to sell a CD with the web site on it for a nominal cost. I only 
set this up less than a week ago, but have already had one taker.

http://www.g8wrb.org/cd/


If that works, it might stop people taking huge amounts - they might 
rather pay the $10 and get one sent to them.

5) You could report someone to their ISP, but its not likely to achieve 
much, unless the same IP is downloading the same material, in which case 
it is a denial of service attack.

6) You might find a lot of the downloads are from Google and other 
search engines. These download documents to index them. Do you really 
need that? Sometimes (e.g. in the case of scanned data sheets) there is 
*nothing* that can be obtained from them.

Hence use a robots.txt file to keep out robots. This does not stop 
anyone getting at data, but most robots will respect this and not bother 
trying to index files you don't want them to.  Here's mine.
webserver2 /usr/local/apache2/htdocs/g8wrb # more robots.txt

User-agent: *
Disallow: /error-messages/
Disallow: /data/Amperex
Disallow: /data/Burle
Disallow: /data/Eimac
Disallow: /data/GEC
Disallow: /data/Machlett_Laboratories
Disallow: /data/Penta
Disallow: /data/Philips
Disallow: /data/RCA
Disallow: /data/Siemens
Disallow: /data/Svetlana
Disallow: /data/Tesla

That basicaly stops search engine robots opening 100's of MB of data 
sheets, most of which are scanned and so contain no text.

7) One option I have thought of is putting a password on a directory, 
but having a web page tell people what username and password to use. 
That will make it hard for someone to copy huge chunks they don't want. 
If they want something they will have to take the trouble to enter the 
username/password, but if they really don't want something, they will 
not bother. You might be able to do that with a .htaccess file, but 
again it depends on how your site is configured.

8) Another option is to put noindex and nofollow in web pages where you 
don't want them indexed. See
http://www.robotstxt.org/wc/meta-user.html

You need to talk to your ISP. If they are not too helpful, drop me a 
line off-list and I can make some more suggestions.

If all the hams got together, they could easily bring your site down 
Rich by doing copying the entire site all at once. Your ISP would take 
you down.

68.0.150.42 - - [22/Jan/2005:23:12:30 +0000] "GET 
/data/Eimac/4CX5000A.pdf HTTP/1.1" 401 1201 
"http://www.g8wrb.org/tetrodes.html"; "Mozilla/4.5 (compatible; HTTrack 
3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:30 +0000] "GET 
/data/Eimac/4CX10000D.pdf HTTP/1.1" 401 1201 
"http://www.g8wrb.org/tetrodes.html"; "Mozilla/4.5 (compatible; HTTrack 
3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:30 +0000] "GET 
/data/Eimac/4CX5000R.pdf HTTP/1.1" 401 1201 
"http://www.g8wrb.org/tetrodes.html"; "Mozilla/4.5 (compatible; HTTrack 
3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:30 +0000] "GET 
/data/Penta/4CX7500A.pdf HTTP/1.1" 401 1201 
"http://www.g8wrb.org/tetrodes.html"; "Mozilla/4.5 (compatible; HTTrack 
3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:31 +0000] "GET 
/data/Eimac/4CX15000A.pdf HTTP/1.1" 401 1201 
"http://www.g8wrb.org/tetrodes.html"; "Mozilla/4.5 (compatible; HTTrack 
3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:31 +0000] "GET /data/Eimac/8295A.pdf 
HTTP/1.1" 401 1201 "http://www.g8wrb.org/pentodes.html"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:31 +0000] "GET 
/data/Eimac/4CX10000J.pdf HTTP/1.1" 401 1201 
"http://www.g8wrb.org/tetrodes.html"; "Mozilla/4.5 (compatible; HTTrack 
3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:31 +0000] "GET 
/useful-stuff/powertrust/?C=N;O=D HTTP/1.1" 200 1860 
"http://www.g8wrb.org/useful-stuff/powertrust/"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:31 +0000] "GET 
/useful-stuff/powertrust/?C=M;O=A HTTP/1.1" 200 1860 
"http://www.g8wrb.org/useful-stuff/powertrust/"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:31 +0000] "GET 
/useful-stuff/powertrust/?C=S;O=A HTTP/1.1" 200 1860 
"http://www.g8wrb.org/useful-stuff/powertrust/"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:32 +0000] "GET 
/useful-stuff/powertrust/?C=D;O=A HTTP/1.1" 200 1860 
"http://www.g8wrb.org/useful-stuff/powertrust/"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:32 +0000] "GET 
/useful-stuff/powertrust/ds_powertrustii-lr.pdf HTTP/1.1" 200 106363 
"http://www.g8wrb.org/useful-stuff/powertrust/"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:32 +0000] "GET 
/useful-stuff/powertrust/hp-ups-manager-swrc24.pdf HTTP/1.1" 200 59388 
"http://www.g8wrb.org/useful-stuff/powertrust/"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:32 +0000] "GET 
/useful-stuff/powertrust/r3000v3.pdf HTTP/1.1" 200 142733 
"http://www.g8wrb.org/useful-stuff/powertrust/"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:44 +0000] "GET 
/useful-stuff/powertrust/r12000v4.pdf HTTP/1.1" 200 125954 
"http://www.g8wrb.org/useful-stuff/powertrust/"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:31 +0000] "GET 
/useful-stuff/powertrust/A3589A.pdf HTTP/1.1" 206 833262 
"http://www.g8wrb.org/useful-stuff/"; "Mozilla/4.5 (compatible; HTTrack 
3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:56 +0000] "GET 
/useful-stuff/powertrust/ug_powertrustii-lr.pdf HTTP/1.1" 200 785431 
"http://www.g8wrb.org/useful-stuff/powertrust/"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:14:47 +0000] "GET 
/useful-stuff/powertrust/ups6kuserguide.pdf HTTP/1.1" 200 943952 
"http://www.g8wrb.org/useful-stuff/powertrust/"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:17:20 +0000] "GET 
/useful-stuff/Sun/cooling-Sun-SPARCstation-20//cool.jpg HTTP/1.1" 200 
45857 
"http://www.g8wrb.org/useful-stuff/Sun/cooling-Sun-SPARCstation-20/"; 
"Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:17:32 +0000] "GET /data/Siemens/ 
HTTP/1.1" 401 1201 "http://www.g8wrb.org/useful-stuff/Sun/firewall/"; 
"Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:17:33 +0000] "GET 
/useful-stuff/Sun/firewall/firewall2.jpg HTTP/1.1" 200 221079 
"http://www.g8wrb.org/useful-stuff/Sun/firewall/"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:18:07 +0000] "GET 
/useful-stuff/Sun/firewall/nofirewall.jpg HTTP/1.1" 200 240773 
"http://www.g8wrb.org/useful-stuff/Sun/firewall/"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:13:05 +0000] "GET 
/useful-stuff/powertrust/ups3kuserguide.pdf HTTP/1.1" 200 1929641 
"http://www.g8wrb.org/useful-stuff/powertrust/"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:12:44 +0000] "GET 
/useful-stuff/powertrust/ug_a1359a.pdf HTTP/1.1" 200 2393005 
"http://www.g8wrb.org/useful-stuff/powertrust/"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:18:42 +0000] "GET 
/y799-amp/closeup-hi.jpg HTTP/1.1" 200 38729 
"http://www.g8wrb.org/y799.html"; "Mozilla/4.5 (compatible; HTTrack 3.0x; 
Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:18:45 +0000] "GET /yagi/pattern.jpg 
HTTP/1.1" 200 27941 "http://www.g8wrb.org/yagi/output.html"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:18:49 +0000] "GET /yagi/ex1.html 
HTTP/1.1" 200 3053 "http://www.g8wrb.org/yagi/optimise.html"; 
"Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:18:49 +0000] "GET /yagi/ex2.html 
HTTP/1.1" 200 4289 "http://www.g8wrb.org/yagi/optimise.html"; 
"Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:18:50 +0000] "GET /yagi/ex3.html 
HTTP/1.1" 200 7335 "http://www.g8wrb.org/yagi/optimise.html"; 
"Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:18:50 +0000] "GET /yagi/ex4.html 
HTTP/1.1" 200 3593 "http://www.g8wrb.org/yagi/optimise.html"; 
"Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:18:51 +0000] "GET 
/yagi/old_sources/yagiuda-1.15.tar.gz HTTP/1.1" 206 27429 
"http://www.g8wrb.org/yagi/"; "Mozilla/4.5 (compatible; HTTrack 3.0x; 
Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:18:55 +0000] "GET /yagi/optimise.1.html 
HTTP/1.1" 200 42482 "http://www.g8wrb.org/yagi/"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:18:51 +0000] "GET 
/yagi/old_sources/yagiuda-1.16.tar.gz HTTP/1.1" 206 125206 
"http://www.g8wrb.org/yagi/"; "Mozilla/4.5 (compatible; HTTrack 3.0x; 
Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:19:00 +0000] "GET /yagi/first.5.html 
HTTP/1.1" 200 680 "http://www.g8wrb.org/yagi/"; "Mozilla/4.5 (compatible; 
HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:19:04 +0000] "GET 
/useful-stuff/powertrust/?C=N;O=A HTTP/1.1" 200 1860 
"http://www.g8wrb.org/useful-stuff/powertrust/?C=N;O=D"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:19:05 +0000] "GET 
/useful-stuff/powertrust/?C=M;O=D HTTP/1.1" 200 1860 
"http://www.g8wrb.org/useful-stuff/powertrust/?C=M;O=A"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:19:06 +0000] "GET 
/useful-stuff/powertrust/?C=S;O=D HTTP/1.1" 200 1860 
"http://www.g8wrb.org/useful-stuff/powertrust/?C=S;O=A"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:19:07 +0000] "GET 
/useful-stuff/powertrust/?C=D;O=D HTTP/1.1" 200 1860 
"http://www.g8wrb.org/useful-stuff/powertrust/?C=D;O=A"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:18:55 +0000] "GET 
/yagi/old_sources/yagiuda-1.17.tar.gz HTTP/1.1" 206 204238 
"http://www.g8wrb.org/yagi/"; "Mozilla/4.5 (compatible; HTTrack 3.0x; 
Windows 98)"
68.0.150.42 - - [22/Jan/2005:23:15:02 +0000] "GET 
/useful-stuff/powertrust/ups12kuserguide.pdf HTTP/1.1" 200 2201154 
"http://www.g8wrb.org/useful-stuff/powertrust/"; "Mozilla/4.5 
(compatible; HTTrack 3.0x; Windows 98)"
webserver2 /usr/local/apache2/logs #

--
Dr. David Kirkby, 
G8WRB

Please check out http://www.g8wrb.org/ 
of if you live in Essex http://www.southminster-branch-line.org.uk/



_______________________________________________
Amps mailing list
Amps@contesting.com
http://lists.contesting.com/mailman/listinfo/amps