Older History of Changes...
Counter was re-released Aug 7/95
LAST DUMP: JUNE 18/96:
Jan 26: (an all-nighter)
After a week on the NEW HONKIN' SERVER, we were getting FLOODED with hits by one user (250,000 per day). The amount of acitivity on the disk rendered the NFS-mounting useless (Geo-counts were on the same volume as the standard daemon's access_
log files). I tried to compensate by configuring an FDDI card on the new machine, but it matters worse --- it just got "no NFS volume available" messages faster.
I migrated all of the log files from the old server to the new one, which took time. Then, the volume wasn't big enough, so I devised a new partitioning scheme that could have either sped things up, or broke things for good.
The STATS pages will have to stay broken for a few days, but I think we're back in business. Email me if you have noticed any difference one way or the other.
Stopped using the FDDI. For some reason, it was KILLING the performance.
In doing so, I lost DNS control for the night, and your "last N Users" are only listed by IP address - not their names. Sorry 'bout that. Should be fixed now.
Fixed the nightly stats/fill program that checks references per day, URLs etc., so the stats should be more up-to-date.
I inadvertently removed my gif conversion libraries, while re-positioning some disk space (I'm running out). From Mar4-6pm to Mar5-9:30am, the counter was working, just not displaying properly. It should be fixed now. I wonder why I only got two email messages on this.
Feb 14/95 DUMP:
Changed the action of &full.
&full Will now give you a "stats menu" of what you can look at.
I've built a rather innefficient method for subdividing the domains, based on what level you want.
For an example machine name of
a &level1 will display stats based on "ca"
a &level2 will display stats based on "ca.ualberta"
a &level3 will display stats based on "ca.ualberta.ucs"
anything else will display stats based on the entire domain, basically mimicking the "old" &full.
Bug in my backup techniques for keeping a busy counter in sync.
Normally, when a file-locking problem causes the counter to "seem like zero", I double-check the count with the "daily stats" file. If that file is busy, I count the number of lines in the "extended log (last n users)". This doesn't work all that
well when extended logs aren't being kept (ref per day>100).
For "no extended stats" counters, I only check "daily", and if that doesn't go, I abort the count, and you get the "broke" gif for this one "locked" access.
A Christmas present: Extra logs done. "Top N Users", "Top N Domains", etc.
To use, either go into the Housekeeping, and press buttons, or add a number (like 20) to the end of a leveln, like this:
&level1:10 will show top 10 countries (here is an
&level2:15 will show top 15 companies (here is an
&level3:15 will show top 15 subdomains (here is an
&frequent:20 will show top 25 individuals (here is an
I expired all counters that haven't been referenced in a month. That includes the "registered" ones that haven't been referenced in a month, too.
Added a button to the "Inspector" in housekeeping which can be used to re-synchronize the "daily total" count with the "normal" count. This is better than a reset, which wipes out the daily stats. Use this when you feel the count that is being displ
ayed is different than what "daily" is reporting. NOTE: This will set both of these numbers to the SAME value, that being the HIGHER of the two. -- Nothing else will be changed.
NEW HONKIN' SERVER added to handle the system load. Site was up to 200,000 big cgi-script hits per day with basically 1 server. Although a little premature to tell, the system load seems to now be much more improved/balanced.
- Update Jan 21: New machine doesn't handle perl "unlink" properly. Broken gifs resulted. I think I've fixed
it as of 5pm.
Dec 20/95 Dump:
- Made a modification to the Registration program that lets you "claim"/"inherit" a non-registered counter. It should preserve the previous counts, create date, logs, etc.
- Migrated the Counter program (and its DBM databases) to a bigger server.
Caused two hours of down time. Although this was extremely tough, because of the re-fitting of little-endian data onto a big-endian machine, everything should be transparent, but if you notice problems that have cropped up because of this significant change, EMail me. (I'd also be interested to know if you're now getting better performance)
Registration program broke. File & PATH permissions. Fixed around 11:30am.
ARGGH. Some people read too well. If you have references to Geo-counter.new -- get rid of them. Also, working on a fix to the "stats" pages.
<!--------- Some comment --------->
<!-- ======== Some comment ======= -->
Sorry about that!
I fiddled with the names of some of the fonts. It was dumb to have some "Mediums" largers than some of the "Bigs". So, some people might have to adjust their font names. I won't do this too often - I just thought this would be better. See http://www.ualberta.ca/htbin/showfonts
- Counter has been re-released.
- Several restrictions have been added. (See notes)
- For a background on the progress of these changes, see Geo-counter Progress page
- Change to the algorithm for Histograms to be a little nicer at scaling.
- Most data on server moved to shared space to prepare for move to mutiple-cpu service.
- Tightened a few of the blacklist algorithms.
- Bug fixed: Reporting of advanced stats was being turn off prematurely for registered counters. Should now properly report if RPD<100, rather than 50.
I had to deal with a couple of really snide people about this, who, rather than saying, "Geo, I'm at 70rpd and registered, and it's not letting me see the advanced stats", I was getting, "Hey. How come I can't see my advanced stats!?" Click here for my response
- Bug fixed: "Reset to ZERO" under registration procedure actually only reset the attributes, but not the count. That part now works.
- Produce better error messages when you are asking for stats on a counter that isn't available.
- Bug fixed: "IGNORE LIST" under housekeeping procedure was not working for a few people. Turns out they were adding unprintable characters at the end of the lists of domain names. (Like using a TAB to go to the next field when the browser can't
handle it). This was hard to track down, but I think I've devised a way to ignore those unprintables. --- The real answer was/is to re-type the entries.
- I'm also bowing to pressure (from too many people who can't read), and changing the "Not now-Try Later" GIF to include the fixed times 8am-5pm Edmonton Time, even though that means that if I change
the restricted hours they wan't be reflected in the GIF. Ie, so much for staying generic for portability.
Below this line are really old changes
- Expired counters not referenced since May 18
- made the "bad counter reset" a nightly cron-job.
May 15/95 Resetting URLs
May 11/95 Bug report:
- Implemented an automatic "reset" of the URL database.
- One chance in 200 that the URL list will be reset itself, and lowered the chances that a new URL will "pop" into the list from 10% to 3%, hopefully minimizing the FAQ 2 email questions.
May 9-10, 1995
- I'm still tracking down the following problems:
- Transparent GIF filter causes server to crash occasionally
I've disable transparency for the time being...
Chose one of the other fonts for a while, if you don't want to "put up with the ugliness"
- Symptom: A few *HEAVILY* used counters keep getting reset to 1
File locking thru Perl isn't as good as
advertised, and I'm struggling with making it better.
Heavily used (> 20 access per minute) pages are the only
counters prone to this behaviour.
ALL YOUR DATA is *still there* - you just have to look at it
"manually" by using the &all parameter.
****Update: May 12**** While I still can't figure out how to
solve this, I built a kludge to re-count the log files, so the
counters will show up "right" again. I will re-run this script
occasionally, to ensure that the counts are right for the
May 4, 1995
GIF IMAGES ARE AVAILABLE!!
Lots of font choices!! About Time!! Hooray!!
Change all references to the program from:
After that, by default, you'll get a font that looks like this:
You can change your counter's look by adding the phrase "&font:Size-style" to the end of the request.
There are lots of font choices. So, here's what your counter request should look like to get the Medium-green characters:
Note that it takes longer to load gifs, because there's more info. If you want fast, continue using ".xbm"
May 4, 1995
- MEANINGFUL NAME CHECKING
I've added some error checking when the program encounters a new counter request. If you follow the new MEANINGFUL EXAMPLES page, you can get an idea of what to expect when you encounter an error.
May 8, 1995
May 4, 1995
- Time Zone Info Added
Added a quick-fix to tell people that times are reported in MDT. A more permanent solution will exist when I build a registration system. See FAQ 4
- CLEANUP & DISK MOVE
I've just about run out of disk. I've moved the counters to a new volume, but that's temporary, too! I'm getting about 90 new counters A DAY!!
- Because of the migration, some of the activity between 8pm-10pm Edmonton Time were lost. Sorry 'bout that.
- I also expired counters not referenced since 9500420. That made 286 abandoned counters in one month.
- I (hopefully) corrected a slight problem in analyzing which domain names to reverse.
April 28, 1995
Mar 28, 1995
- CLEANUP DONE
"Ownership" URLs erased, but will re-populate themselves as appropriate browers can tell me what the web page was that called the counter. (See FAQ 2 for how/why this happens)
- Any log file that has not been referenced since 950324 has been removed.
- Any file with bizarre characters or odd file names was removed. (These include ones that started with a dash, or had a '>' in it). Files with special characters, or stupid names will continue to be periodically eradicated. (See Meaningful Names Examples for more details).
Mar 12, 1995
- Reversed domains didn't make any sense for numbers, (duh) , so a hook was put in to preserve numeric ip addresses in "&full" stats. To properly reverse-order the legacy of data, the server was "hiccupping" for about 20 minutes between 2:30 and 3:00pm - some files may have lost a few hits.
- After 12500 hits, one of the counters seemed to reset. This random occurrence might be related to the heavy usage of the file. I've added extra "locking" checks during file-opening procedures to hopefully compensate for this weird thing. (Funny - I thought Perl was supposed to manage these suckers for me)
Mar 8, 1995
- Small bug fix lets users use tildes in their chosen log names. Note that this may effect some counters that had special characters (especially backslashes).
- Built a hook to record "first use"/"create date" of the counter. Retrieved only in "log" printouts, currently.
Mar 7, 1995
- Modification to filter out "xxxyyy.html#anchor" and imagemap "xxxyyy.blah?123,456" urls from ownership registries. "Owner" databases reset, but should re-fill themselves quickly.
Feb 28, 1995
- Changed the "date" reporting so it doesn't take up as much room (or disk space)
- Printouts now contain the URL(s) -if known- that reference the counter
- Format of Printout revised to have a better preformatted appearance
- Add the "
"full domain stats" option
This is a new report that shows sorted statistics based on the
machine name (by reversed domain name). One stats line per visitor.
This is something I've wanted for a while, so it's done. Note,
however, you may not want to add this "full view" to the average user.
Just know how to reference it yourself, perhaps?
See this new
hunk of html code that you can add to your
file to give you "full" stats.
Made a linked synonym of "Geo-counter" to "Geo-counter.xbm"
Feb 28, 1995
This is obsolete.
Some browsers can indeed interpret inline X-bitmaps, but instead of
looking at the mime content look at suffix of the path to the script.
Change all references to the program
to allow more client browsers to properly view the counter.
Made a hook to track signatures of the web documents that use the Geo-counter program. This hook will record "collisions" but will not police two html documents "sharing" the same log files.
- Some browsers can't handle XBM graphics.
- Sending a graphic is slower than sending 3 or 4 digits.
- The visual "look" of the counter output does not necessarily "fit in nicely" with the rest of your page.
- I'm thinking of switching to GIF, if I can figure out how to do it. (Done: May 4/95)
- XBM is a text-only stream. It's quite small, but still hundreds of characters. I'll see what I can do. (Done: May 4/95)
- I've built the program to be flexible (sort of), and in the future, I'll be giving CHOICES of different charactersets to use. (Done: May 4/95)
This seems to no longer be an FAQ, so it's been moved here.
Q 2: URLs Referencing the log
Would you give me a
quick explanation regarding the "Log referenced by URL:" part of
the tail log?
I made a point of using the entire path to each
document that calls your program because I didn't want other people
to be using the same variable (I notice two people seem to be
calling the Geo-counter with index.html; I wanted my counter
variable to be unique so I used a full path). However, much to
my surprise, today when I checked the tail log of one of my
discovered that listed under "Log referenced by URL:" was
http://home.mcom.com/home/welcome.html (the Netscape home page)
Any ideas as to why that showed up as a referenced URL? I'm
sort of baffled
Unique names are good. I like unique names.
The "Referenced by: ..." stuff takes a little explaining.
Some, but not all browsers (ie your desktop app, such as Mosaic, WinWeb, Netscape) tell the Web Server (httpd daemon) extra environment variables, one of which is the variable HTTP_REFERER -- this variable tells the httpd what URL asked for the new resource (eg an XBM file). For a little info on this, see http://www.ualberta.ca/WebSupport/DumpEnvironment.html
Because I do not have a registration procedure for this program (hence the "pick your UNIQUE NAME, and be polite"), I have to rely on someone else to provide me with the name of the URL that called the logger. This only happens when *any* user with the HTTP_REFERER visits the page (remember, there are no counter "owners" -- be polite).
I have noticed that some browsers that provide the HTTP_REFERER variable get confused once in a while, and actually don't remember where they truly came from. My suspicion is that it happens when a user calls a page directly from his hotlist, after JUST launching the app.
Oh well. It's just a courtesy piece anyway. I thought it would help to alert any individual that someone else "wasn't being polite", and cloned your counter name.
The files that contain the HTTP_REFERER URLs can be wiped out, and my program will "re-generate" the list, as new visitors arrive. I'm thinking of making this a monthly house-keeping chore.