[Adium-devl] xml format possiblities and drawbacks

[Graham Booker]

Speaking from experience here (I did the XML logging in Fire after  
reading the code in Colloquy):

1) Solved by incremental updates to the XML log file.  We already  
have a solution for this in place in Adium through the AIXMLAppender  
class.  This doesn't require reading the entire log from disk, and  
writing the entire new log to disk for each message, unlike what  
Kopete was doing.  Fire uses incremental updates and I can  
conclusively say that I have never noticed a single slowdown.   
Colloquy is the same, and I have log files there in the several  
megabyte range.

2) I think the main gripe here is viewing the file outside the app.   
I would recommend a web browser for this.  In Fire, we didn't have a  
single gripe about the XML log format over the simpler text based  
format.  Also, conversion to text is a simple operation, and grep  
still works on the XML files.

3) The main gripe here (about XML) is that the history used a  
different XML format than the internal XML format that was fed  
through the XSLT for display.  This meant that there were essentially  
two XML formats to deal with, one for the display, and one for the  
history.  The solution is to use the same format for both, which is  
what we did in Fire (and so does Colloquy).  Adium doesn't currently  
use an XML format to feed through an XSLT for display, so this  
doesn't apply.  If it were to in the future, using the logging format  
is a natural choice unless there is an exceptional reason to not use it.

4) The gripe here is search speed.  We get around much of that  
through Spotlight, but for those without spotlight, it is quite  
trivial to search for content very quickly in XML.  Simply toss all  
the tags and fix the XML entities.  This will give you the plain text  
of the log to search, and is a much cheaper operation than XML  
parsing.  The is the mode we used in Fire, and the slowdown for a  
linear search through files (non-spotlight) was slightly slower, but  
not noticeable over parsing an HTML log (since you have to do the  
exact same thing there too).

Hope this helps put some fears to rest.

- Graham


On May 23, 2006, at 9:54 PM, Matt Rogers wrote:

> Hi all,
>
> I heard from one of the gaim guys that y'all were working on a new  
> XML format
> for the logs. I was talking with Catfish_Man about some of the  
> problems
> Kopete has had with the use of XML in the log files. Here's 4 bugs  
> that I
> found that express some of the difficulties users have had with the  
> log file
> format and it's resulting implementation in Kopete. I'm sure we can  
> do it
> better than we are, so this is just food for though for the  
> discussion.
>
> http://bugs.kde.org/show_bug.cgi?id=65982 - updating of chat window  
> takes ages
> when there are many messages
>
> http://bugs.kde.org/show_bug.cgi?id=89744 - request to change  
> history format
> to plain text for easy searching from the command line
>
> http://bugs.kde.org/show_bug.cgi?id=94178 - Kopete's chat log  
> system is
> confusing (talking about on-disk layout mostly)
>
> http://bugs.kde.org/show_bug.cgi?id=113794 - History viewer is slow
>
> Again, this is just food for though for the discussion. Learning  
> from our
> mistakes and all that. :)
> --
> Matt
> (Please CC any replies, i'm not subscribed)
>
> _______________________________________________
> Adium-devl mailing list
> Adium-devl at adiumx.com
> http://adiumx.com/mailman/listinfo/adium-devl_adiumx.com
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2415 bytes
Desc: not available
URL: <http://adium.im/pipermail/devel_adium.im/attachments/20060523/48ef96f9/attachment.p7s>