# Log Analysis

**URL:** <https://community.wowza.com/t/log-analysis/129>\
**Category:** Wowza Streaming Engine\
**Created:** [March 12, 2007, 1:28pm UTC](https://community.wowza.com/t/log-analysis/129 "2007-03-12T13:28:49Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/flex002/user_avatar/community.wowza.com/system/32/447_2.png) [@system](https://community.wowza.com/u/system)\
**Post date:** [March 12, 2007, 1:28pm UTC](https://community.wowza.com/t/log-analysis/129/1 "2007-03-12T13:28:49Z")

</div>

Can anyone recommend a log analyzer that will process Wowza/FMS logs?

Thanks,

Penneths

---

<div class="post-metadata">

**Author:** ![James\_Cunningham](https://avatars.discourse-cdn.com/v4/letter/j/b2d939/32.png) [@James\_Cunningham](https://community.wowza.com/u/James_Cunningham)\
**Post date:** [March 12, 2007, 5:19pm UTC](https://community.wowza.com/t/log-analysis/129/2 "2007-03-12T17:19:01Z")

</div>

I use awstats:

I have spent quite a lot of time on this, and Charlie really helped out when creating Wowza’s logs in the first place. I’m not sure what FCS2 is like, but 1.5 consisted of some weird ascii dump which was impossible to work with…

I really only wanted to know which files had been played and the amount of data transfer for on-demand files which I have got working really well (see below). It will still work for live stuff. As for analysing more interactive stuff, that will be a bit more difficult.

Anyway here is what I do:

I edit the log4jproperties file as follows:

> log4j.appender.R=org.apache.log4j.DailyRollingFileAppender
> 
> log4j.appender.R.DatePattern='.'yyyy-MM-dd
> 
> log4j.appender.R.File=D:/logs/wowza/wowzamediaserver\_access.log
> 
> log4j.appender.R.layout=com.wowza.wms.logging.ECLFPatternLayout
> 
> log4j.appender.R.layout.OutputHeader=true
> 
> log4j.appender.R.layout.QuoteFields=false
> 
> log4j.appender.R.layout.Delimeter=tab
> 
> log4j.appender.R.layout.Fields=c-ip,date,time,x-sname,x-status,c-proto,sc-stream-bytes
> 
> log4j.appender.R.layout.CategoryInclude=stream
> 
> log4j.appender.R.layout.EventInclude=destroy

I am only logging the fields that I specifically want, and only doing so for the category stream at the point when it is destroyed. This provides us with a simple line in the log file with the important information:

> 192.168.1.70 2007-03-08 18:54:06 demo/merry\_melodies\_falling\_hare 200 rtmpt (HTTP-1.1)

I have been streaming via rtmpt over port 80 to get around firewalls and the space in **rtmpt (HTTP-1.1)** may cause an issue if you do not use a tab as the log delimiter, so best keep it as that!

Next I install [awstats](http://awstats.sourceforge.net/). this will require a Perl interpreter (If you are using Windows then I recommend ActivePerl).

You then need to set up a config file for awstats. (Copy and paste the demo one and edit it). The important bits are:

> LogType=S
> 
> LogFormat=“c-ip date time cs-uri-stem c-status protocol sc-bytes”
> 
> ValidHTTPCodes=“200 304 408”

(I add 408 so that interrupted streams still log the data transfer)

And if you are analysing multiple file on a Windows server then

> LogFile=““C:/Perl/bin/perl.exe” “d:/Program Files/AWStats/tools/logresolvemerge.pl” d:/logs/wowza/\* |”

The double quotes took me ages to work out!!!

Finally you will need to edit the awstats.pl file as it does not know what to do with the rtmp(t) protocols and throws up errors unless you include them as options in line 6338.

Just add the extra OR options:

> || $field[$pos\_method] eq ‘rtmp’ || $field[$pos\_method] eq ‘rtmpt (HTTP-1.1)’

Assuming you’ve set up your cgi-bin properly and filled any little gaps that I may have left, you should be able to get your stats by going to:

http:///cgi-bin/awstats.pl?config=

Hit the update link and you will have instant stats!!!

I have spent quite a lot of time editing the config file as there are lots of irrelevant bits in there, and am really happy with the results.

Hope that all makes sense - it did take me a good week to sort it all out!

---

<div class="post-metadata">

**Author:** ![Charlie\_Good](https://sea2.discourse-cdn.com/flex002/user_avatar/community.wowza.com/charlie_good/32/383_2.png) [@Charlie\_Good](https://community.wowza.com/u/Charlie_Good)\
**Post date:** [February 20, 2008, 5:48am UTC](https://community.wowza.com/t/log-analysis/129/3 "2008-02-20T05:48:07Z")

</div>

I have not done any work with awstats. Hopefully someone else on the forums can help.

Charlie

---

<div class="post-metadata">

**Author:** ![James\_Cunningham](https://avatars.discourse-cdn.com/v4/letter/j/b2d939/32.png) [@James\_Cunningham](https://community.wowza.com/u/James_Cunningham)\
**Post date:** [February 20, 2008, 2:07pm UTC](https://community.wowza.com/t/log-analysis/129/4 "2008-02-20T14:07:52Z")

</div>

I can’t remember what the c-proto field is, but it looks like that’s where your problem is.

You need to make sure that your logfiles from Wowza and the “Logformat” string are consistent.

I remember encountering similar problems, but stripped out everything that was not important and substituted some “dummy” variables just to keep awstats happy, and then added in the important data that I needed once that was working…

---

<div class="post-metadata">

**Author:** ![James\_Cunningham](https://avatars.discourse-cdn.com/v4/letter/j/b2d939/32.png) [@James\_Cunningham](https://community.wowza.com/u/James_Cunningham)\
**Post date:** [March 1, 2009, 12:53pm UTC](https://community.wowza.com/t/log-analysis/129/5 "2009-03-01T12:53:24Z")

</div>

Hi Chaso,

Yes that does sound familiar, although it’s over a year ago and I can’t remember exactly what I did to fix it. Have you followed the instructions that I set up?

Looking at it, it’s probably because awstats does not recognise rtmp or rtmpt by default so you will have to add those into the awstats.pl line. It was line 6338 a year ago, but I expect this has changed with a newer version.

Look for $field[$pos\_method] and add the extra OR clauses:

|| $field[$pos\_method] eq ‘rtmp’ || $field[$pos\_method] eq ‘rtmpt (HTTP-1.1)’

Jim

---

<div class="post-metadata">

**Author:** ![PENNY\_BOYS](https://avatars.discourse-cdn.com/v4/letter/p/ecb155/32.png) [@PENNY\_BOYS](https://community.wowza.com/u/PENNY_BOYS)\
**Post date:** [March 12, 2007, 5:39pm UTC](https://community.wowza.com/t/log-analysis/129/6 "2007-03-12T17:39:08Z")

</div>

Jim,

Many thanks for your help. So far I’m impressed by Wowza and would rather not use FMS. I’ll follow all the steps in your post and see if I can get it to work.

Thanks again,

Penneths

---

<div class="post-metadata">

**Author:** ![Richard\_Lanham](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@Richard\_Lanham](https://community.wowza.com/u/Richard_Lanham)\
**Post date:** [February 12, 2010, 12:31pm UTC](https://community.wowza.com/t/log-analysis/129/7 "2010-02-12T12:31:51Z")

</div>

This line is not in the access appender, try commenting it out:

```auto
log4j.appender.serverStats.layout.CategoryInclude=session,stream

```

Richard

---

<div class="post-metadata">

**Author:** ![Richard\_Lanham](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@Richard\_Lanham](https://community.wowza.com/u/Richard_Lanham)\
**Post date:** [August 11, 2010, 12:19pm UTC](https://community.wowza.com/t/log-analysis/129/8 "2010-08-11T12:19:56Z")

</div>

You can modify your Fields list. You have this:

```auto
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_stats.layout.Fields=x-severity,x-category,x-event;date,time,c-client-id,c-ip,c-port,cs-bytes,sc-bytes,x-duration,x-sname,x-stream-id,x-spos,sc-strea
m-bytes,cs-stream-bytes,x-file-size,x-file-length,x-ctx,x-comment

```

The superset of available log Fields is in the log4j.properties file comment:

```auto
# Field list
#date,time,tz,x-event,x-category,x-severity,x-status,x-ctx,x-comment,x-vhost,x-app,x-appinst,x-duration,s-ip,s-port,s-uri,c-ip,c-proto,c-referrer,c-user-agent,c-client-id,cs-bytes,sc-bytes,x-stream-id,x-spos,cs-stream-bytes,sc-stream-bytes,x-sname,x-sname-query,x-file-name,x-file-ext,x-file-size,x-file-length,x-suri,x-suri-stem,x-suri-query,cs-uri-stem,cs-uri-query

```

Richard

---

<div class="post-metadata">

**Author:** ![Richard\_Lanham](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@Richard\_Lanham](https://community.wowza.com/u/Richard_Lanham)\
**Post date:** [July 11, 2011, 9:53am UTC](https://community.wowza.com/t/log-analysis/129/9 "2011-07-11T09:53:15Z")

</div>

Please tell us what you are trying to do, and what does not work.

Richard

---

<div class="post-metadata">

**Author:** ![Richard\_Lanham](https://avatars.discourse-cdn.com/v4/letter/r/c68b51/32.png) [@Richard\_Lanham](https://community.wowza.com/u/Richard_Lanham)\
**Post date:** [October 19, 2011, 1:11pm UTC](https://community.wowza.com/t/log-analysis/129/10 "2011-10-19T13:11:57Z")

</div>

Here is an article on using AWStats and Wowza:

[http://www.wmconsulting.info/awstats-mod-for-process-wowza-stats/](http://www.wmconsulting.info/awstats-mod-for-process-wowza-stats/)

And there is some discussion here that might be useful that is easy to find if you search for “Wowza AWStats”

Richard

---

<div class="post-metadata">

**Author:** ![Jure\_Raicevic](https://avatars.discourse-cdn.com/v4/letter/j/a87d85/32.png) [@Jure\_Raicevic](https://community.wowza.com/u/Jure_Raicevic)\
**Post date:** [February 20, 2008, 1:41pm UTC](https://community.wowza.com/t/log-analysis/129/11 "2008-02-20T13:41:51Z")

</div>

I have problems with this setup. My problem is with editing the awstats.pl file.

I’m not sure if I set the line specifying fields inside log file:

> || $field[$pos\_method] eq ‘rtmp’ || $field[$pos\_method] eq ‘rtmpt (HTTP-1.1)’

when I run perl update, all of my lines inside log files get dropped:

> perl awstats.pl -config=test2 -update -showdropped

So I get the next response:

> C:\Inetpub\SPS\_stat\cgi-bin\>perl awstats.pl -config=test2 -update -showdropped
> 
> Create/Update database for config “./awstats.test2.conf” by AWStats version 6.7
> 
> (build 1.892)
> 
> From data in log file "C:\Documents and Settings\jurera\Desktop\testni\_wowza.log
> 
> "…
> 
> Phase 1 : First bypass old records, searching new record…
> 
> Searching new records from beginning of log file…
> 
> Dropped record (method/protocol ‘c-proto’ not qualified when LogType=S): #Fields
> 
> : c-ip date time x-sname x-status c-proto sc-stream-bytes
> 
> Jumped lines in file: 0
> 
> Parsed lines in file: 5
> 
> Found 1 dropped records,
> 
> Found 4 corrupted records,
> 
> Found 0 old records,
> 
> Found 0 new qualified records.
> 
> Has anybody come up with the same problem?

---

<div class="post-metadata">

**Author:** ![Joshua\_Rubingh](https://avatars.discourse-cdn.com/v4/letter/j/ba8739/32.png) [@Joshua\_Rubingh](https://community.wowza.com/u/Joshua_Rubingh)\
**Post date:** [July 20, 2008, 9:03pm UTC](https://community.wowza.com/t/log-analysis/129/12 "2008-07-20T21:03:36Z")

</div>

Hi,

Great info. I am already familair with AWStats, so that helps. The problem that is here going on are two things.

1. The logfile in wowza is tab delimeted, while AWStats defaults thinks that is space delimeted. So you have to edit the settings **LogSeparator** to ‘\t’

That wil fix the corrupted records.

1. The problem that then rises is that when you use rtmp streaming, it is not a valid protocol. AWstats will reject all lines with the rtmp as protocol.

To fix this, you edit the awstats.pl file from AWStats. Go to line +/-6400. There should be the line:

```auto
elsif (($LogType eq 'W' || $LogType eq 'S') && ($field[$pos_method] eq 'GET' || $field[$pos_method] eq 'mms' || $field[$pos_method] eq 'rtsp' |
| $field[$pos_method] eq 'http' || $field[$pos_method] eq 'RTP' )) {

```

add the following to the if statement:

```auto
|| $field[$pos_method] eq 'rtmp'

```

so it will become:

```auto
elsif (($LogType eq 'W' || $LogType eq 'S') && ($field[$pos_method] eq 'GET' || $field[$pos_method] eq 'mms' || $field[$pos_method] eq 'rtsp' |
| $field[$pos_method] eq 'http' || $field[$pos_method] eq 'RTP'|| $field[$pos_method] eq 'rtmp')) {

```

So AWStats will not reject lines with the rtmp protocol. And your done!

There will be one dropped line, and that is the header line which is ok.

I hope this solves the problems and complete this little HOWTO of streamingwizard

TheYOSH

---

<div class="post-metadata">

**Author:** ![Charles\_Hockenbarger](https://avatars.discourse-cdn.com/v4/letter/c/2bfe46/32.png) [@Charles\_Hockenbarger](https://community.wowza.com/u/Charles_Hockenbarger)\
**Post date:** [February 27, 2009, 4:12pm UTC](https://community.wowza.com/t/log-analysis/129/13 "2009-02-27T16:12:14Z")

</div>

Jim and/or Yosh:

Did Awstats complain to either of you about not having a %method or %methodurl variable in your LogFormat string?

I’m encountering it giving me an error that it needs the %method in the string, but it doesn’t seem to matter where I put it, it doesn’t like it as a dummy variable.

Thanks!

---

<div class="post-metadata">

**Author:** ![Alessandro\_Ranellucc](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@Alessandro\_Ranellucc](https://community.wowza.com/u/Alessandro_Ranellucc)\
**Post date:** [May 7, 2009, 10:17am UTC](https://community.wowza.com/t/log-analysis/129/14 "2009-05-07T10:17:11Z")

</div>

To avoid modifying AWStats, you can use the following line in the config:

```auto
LogFormat="cat /path/to/logs/*.log | sed -r 's/rtmp(t \(HTTP-1.1\))?/http/' |"

```

---

<div class="post-metadata">

**Author:** ![Alejandro\_Ferrari](https://avatars.discourse-cdn.com/v4/letter/a/d6d6ee/32.png) [@Alejandro\_Ferrari](https://community.wowza.com/u/Alejandro_Ferrari)\
**Post date:** [August 11, 2010, 12:06am UTC](https://community.wowza.com/t/log-analysis/129/15 "2010-08-11T00:06:32Z")

</div>

Hi, fo use AWStats and support all the protocols need to do some changes into the awstats.pl

I have this running with the package modified. I’ll try to post the instruction to do this during the Week.

Regards,

Alejandro

---

<div class="post-metadata">

**Author:** ![Alejandro\_Ferrari](https://avatars.discourse-cdn.com/v4/letter/a/d6d6ee/32.png) [@Alejandro\_Ferrari](https://community.wowza.com/u/Alejandro_Ferrari)\
**Post date:** [February 1, 2011, 4:51pm UTC](https://community.wowza.com/t/log-analysis/129/16 "2011-02-01T16:51:03Z")

</div>

Hi Guys,

Here I have make public the latest mod for Awstats and Wowza Logs, please try this and send me any issue, all the feedback is appreciated…

[http://www.wmconsulting.info/awstats-mod-for-process-wowza-stats/](http://www.wmconsulting.info/awstats-mod-for-process-wowza-stats/)

This mod is based into the latest Awstats version with some extra lines for support all the protocols and Live Stream… I’ll add the changelog soon…

Regards,

Ale

---

<div class="post-metadata">

**Author:** ![Alejandro\_Ferrari](https://avatars.discourse-cdn.com/v4/letter/a/d6d6ee/32.png) [@Alejandro\_Ferrari](https://community.wowza.com/u/Alejandro_Ferrari)\
**Post date:** [July 7, 2011, 9:00am UTC](https://community.wowza.com/t/log-analysis/129/17 "2011-07-07T09:00:44Z")

</div>

what error do you have?

---

<div class="post-metadata">

**Author:** ![Alejandro\_Ferrari](https://avatars.discourse-cdn.com/v4/letter/a/d6d6ee/32.png) [@Alejandro\_Ferrari](https://community.wowza.com/u/Alejandro_Ferrari)\
**Post date:** [October 20, 2011, 12:45am UTC](https://community.wowza.com/t/log-analysis/129/18 "2011-10-20T00:45:42Z")

</div>

Hi Ban\_ban,

the Dropeed lines is normally why your line sintax is not the same than awstats have setting.

add to your line the params: -showdropped

this will show you the dropped lines, after that the unique way to see what happend is compare the awstats parser line with the wowza logs, possible you have some param more or into other order.

Are you using RTMP only or are make the tests with HLS or RTSP?

Paste some lines here I’ll try to help…

Regards,

Alejandro

wmconsulting.info

---

<div class="post-metadata">

**Author:** ![Leigh\_Spiegel](https://avatars.discourse-cdn.com/v4/letter/l/7c8e57/32.png) [@Leigh\_Spiegel](https://community.wowza.com/u/Leigh_Spiegel)\
**Post date:** [February 12, 2010, 9:21pm UTC](https://community.wowza.com/t/log-analysis/129/19 "2010-02-12T21:21:16Z")

</div>

Minor correct to post about log modified so you don’t have to hack awstats:

Should read LogFile= instead of LogFormat

```auto
LogFile="cat /path/to/logs/*.log | sed -r 's/rtmp(t \(HTTP-1.1\))?/http/' |"

```

---

<div class="post-metadata">

**Author:** ![Leigh\_Spiegel](https://avatars.discourse-cdn.com/v4/letter/l/7c8e57/32.png) [@Leigh\_Spiegel](https://community.wowza.com/u/Leigh_Spiegel)\
**Post date:** [February 12, 2010, 9:53pm UTC](https://community.wowza.com/t/log-analysis/129/20 "2010-02-12T21:53:26Z")

</div>

I’ve set my logging up differently as follows:

```auto
log4j.logger.${com.wowza.wms.context.VHost}.${com.wowza.wms.context.Application}=INFO, ${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_access, ${com.wowza.wms.context.VHost}_${com.wowza.w
ms.context.Application}_error, ${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_stats

log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_access=org.apache.log4j.DailyRollingFileAppender
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_access.DatePattern='.'yyyy-MM-dd
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_access.File=${com.wowza.wms.ConfigHome}/logs//${com.wowza.wms.context.Application}/wowzamediaserver_access.log
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_access.layout=com.wowza.wms.logging.ECLFPatternLayout
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_access.layout.Fields=c-ip,date,time,x-sname,x-status,c-proto,sc-stream-bytes
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_access.layout.OutputHeader=true
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_access.layout.QuoteFields=false
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_access.layout.Delimeter=tab
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_access.layout.CategoryInclude=session,stream
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_access.layout.EventInclude=destroy

```

I notice in the main access.log it shows streams from an iphone…

However in the above setup it doesn’t? Can’t seem to work out why any ideas?

[Next page](https://community.wowza.com/t/log-analysis/129.md?page=2)