PRODUCT






Home









Free Download








Installation Instructions





FAQ





FAQ








Ask A Question





LEARN SCRIPTING





Overview








Lesson 1








2


3


4


5








Exam





SAMPLE SCRIPTS





Computer








Internet








Administrators








Developers








Data








Miscellaneous





HELP / DOCUMENTATION





Commands








Automated Internet








Automated Editors








Sample Scripts








Precompiled Functions








System Features






  Sample Script - SS_WebLogParser

( Some of the sample scripts may not be reproduced correctly in html because those scripts, especially web-related scripts, have html tags such as < tr > in their code.

For an accurate, copy-and-paste'able text version of this script, see SS_WebLogParser.txt . )



#####################################################################
# SCRIPT: SS_WebLogParser
#
# This script is a sample Web Log Parser. It takes a web log file and and
# reports pages, hit counts, referrers and referrer counts.
#
# The name of the file containing the web log is passed using FVA (Forward
# Variable Assignment) for variable $logfile.
#
# The output is written to screen. If your logfile is very large, redirect the
# output using > filename.txt when calling the script.
#
# This script can be stored, and edited as needed, in a file called
# SS_WebLogParser.txt. The script can then be called as
#
# script "SS_WebLogParser.txt" logfile("C:/logs/web/20080820")
#
#####################################################################

# Declare FVA variables.
var str logfile

# Get the contents of the log file into a str variable
var str content
repro $logfile > $content

# We keep the page list in the following variable - it has the format of <r><page><f><count>
# <r> is the marker for the record, <f> is the marker for the field.
var str page_list

# We keep the referrer list in the following variable - it has the format of <r><referrer><f><count>
# <r> is the marker for the record, <f> is the marker for the field.
var str referrer_list

# Parse lines one by one
while ( $content <> "")
do
# Get the next line
var str line
lex -e "1" $content > $line

# Make sure that line is not empty.
if ($line <> "")
do

# The line has the following form.
# 1.2.3.4 x y [20/Oct/2008:01:02:03 -0100] "GET xxx HTTP/1.2" 1 2 "yyy" ...

# The page URL has the following form "GET xxx "
var str page
stex -p -c "^GET ^[" $line > $page
stex "[^ ^" $page > null

# The referrer has the following form "yyy", this is the third double quote in the line
var str referrer
stex "^\"^3]" $line > null
stex -r "]^(\"?)^" $line > $referrer

# Is this page already in the page_list ? If so, we will just increment the count.
# If not, we will add it and set the count to 1.
var str sen_arg
set $sen_arg = "^"+$page+"^"
if ( { sen $sen_arg $page_list } > 0 )
do
# This page is already in page_list.

# Create dynamic argument for the stex command.
var str stex_arg
set $stex_arg = "[^"+$page+"^"

# Get the page record.
var str page_record
stex -p $stex_arg $page_list > $page_record
stex "[^<r>^" $page_record > null

# Extract the count, which is after <f>.
var str count_str
stex -p "^<f>^[" $page_record > $count_str

# Get numeric count
var int count
set $count = makeint(str($count_str))

# Increment count
set $count = $count+1

# Create the updated page record
var str updated_page_record
set $updated_page_record = $page+"<f>"+makestr(int($count))

# Replace page_record with updated_page_record in page_list
var str sal_arg
set $sal_arg = "^"+$page_record+"^"
sal $sal_arg $updated_page_record $page_list > null

done
else
do
# This page is not in the list. Add to the list.
set $page_list = $page_list + "<r>" + $page + "<f>1"
done
endif # Completes if ( { sen $sen_arg $page_list } > 0 )

# The following code is very similar to the above code for page.
# Is this referrer already in the referrer_list ? If so, we will just increment the count.
# If not, we will add it and set the count to 1.
set $sen_arg = "^"+$referrer+"^"
if ( { sen $sen_arg $referrer_list } > 0 )
do
# This referrer is already in referrer_list.

# Create dynamic argument for the stex command.
var str stex_arg
set $stex_arg = "[^"+$referrer+"^"

# Get the referrer record.
var str referrer_record
stex -p $stex_arg $referrer_list > $referrer_record
stex "[^<r>^" $referrer_record > null

# Extract the count, which is after <f>.
var str count_str
stex -p "^<f>^[" $referrer_record > $count_str

# Get numeric count
var int count
set $count = makeint(str($count_str))

# Increment count
set $count = $count+1

# Create the updated referrer record
var str updated_referrer_record
set $updated_referrer_record = $referrer+"<f>"+makestr(int($count))

# Replace referrer_record with updated_referrer_record in referrer_list
var str sal_arg
set $sal_arg = "^"+$referrer_record+"^"
sal $sal_arg $updated_referrer_record $referrer_list > null

done
else
do
# This referrer is not in the list. Add to the list.
set $referrer_list = $referrer_list + "<r>" + $referrer + "<f>1"
done
endif # Completes if ( { sen $sen_arg $referrer_list } > 0 )
done

endif # Completes if ($line <> "")

done # completes do after while ( $content <> "")

# Print the page_list
echo "Pages"
while ( { sen "^<r>^" $page_list } > 0 )
sal "^<r>^" "\n" $page_list > null
while ( { sen "^<f>^" $page_list } > 0 )
sal "^<f>^" "\t" $page_list > null
echo $page_list


# Print the referrer_list
echo "Referrers"
while ( { sen "^<r>^" $referrer_list } > 0 )
sal "^<r>^" "\n" $referrer_list > null
while ( { sen "^<f>^" $referrer_list } > 0 )
sal "^<f>^" "\t" $referrer_list > null
echo $referrer_list

2008-2014, biterScripting.com. All rights reserved.
biterScripting, biterScript, biterBrowser, biterMobile, biterScripting.com, FVA (Forward Variable Assignment) are trademarks of biterScripting.com. Is it biterScripting-compatible ? is a service mark of biterScripting.com. Explorer, Unix, Windows are trademarks, service marks or other forms of intellectual property of their respective owners.