[Adium-devl] Burning Duck 2.0

[Jeffrey Melloy]

I wrote this while at work today:

(I'm at work and adding thoughts throughout the day.  Also, my mail  
is acting up, so I'm going basically from memory.  When GMail is down  
it messes with my world in a very sad way.)

There is a new crash reporter in development.  I've been busy and  
lazy, so it hasn't gotten done.  I'd say it's 80% done to be usable,  
and 50% done to what was envisioned.

It does a couple things.  First, when you import a log, it parses it  
and splits it into the various pieces.  It finds the crashed thread,  
it pulls out all the installed plugins, it parses the OS: 10.4.1  
lines, etc.

After it splits out the crashed thread, it goes line by line and  
inserts the crashed objects into a table with a unique index.  Then  
it takes the object ids and creates what it calls a "crash print".   
The crash print is a string of numbers like  
1-3-42-28-24-21-65-12-62.  Crashes which have the same crash stacks  
will have the same crash print; crashes which have similar crash  
stacks will have similar crash prints.  (IE, the crash print  
28-24-21-65-12-62 will have the same bottom six objects, but the  
first crash will have gotten farther in the stack.)  It creates what  
amounts to a stats table to compare version/crash prints.

I'm very curious how this piece performs.  After it gets more done,  
that might need to be rewritten for speed.

The main page (which I have a screenshot of somewhere, but can't  
find) then reads off that and groups popular crashes by version and  
shows the first half dozen objects in the crash.

That part is done.  The part that isn't done is viewing any crash  
other than the first one in the group, but that won't take much.   
Searching is about 50% done.

There is a logon system envisioned, where guests can see "non- 
sensitive" information (No email, IP, etc) and devs can mark the  
entire group fixed.  How to mark a group fixed needs to have some  
thought put into it.  Grouping itself could use a little thought, too.

As far as the old system goes, the searching has a couple big problems:

1) The version of PostgreSQL used on my server was created in the  
Mesozoic.  It originally communicated through smoke signals, and was  
only recently updated to use the TCP/IP stack.  One of the main,  
documented issues this version has is index bloat on a linearly  
growing table that periodically has old records deleted.  Sound  
familiar?

2) In the SQL Logger, I use a module called tsearch which is  
specifically designed for speedy searching.  I don't on Visual  
Distortion, because it's not installed.  Thus we use the old'n'busted  
method, which is basically a regexp.  A non-indexed regexp.

The new crash reporter will take advantage of tsearch, since to not  
is just silly.

The request was made that we get crashes/day, crashes/version, common  
crashes/version, etc, graphs.  It's on the todo list, but I'm not  
considering them necessary.  However, the new schema is much more  
flexible.

A new host may be nice.  To use the existing codebase, it would  
require PostgreSQL and a servlet engine like Tomcat or Resin.

I apologize for not taking care of some of this stuff sooner; the new  
one is so close it's beyond ridiculous.  When I get home from work I  
have a tendency to not want to do anything on a computer requiring  
thought, and I get busy or lazy or both on the weekends.

Jeff


On Dec 20, 2006, at 3:42 PM, Zac West wrote:

> I'd likely end up writing it, and I have a few ideas in my head  
> about how to index, search and process them quickly.
>
> Zac
> _______________________________________________
> Adium-devl mailing list
> Adium-devl at adiumx.com
> http://adiumx.com/mailman/listinfo/adium-devl_adiumx.com