[Adium-devl] Java-based libraries... a ponder, a thought, an apology

[Graham Booker]

Since I spent a lot of time messing with libfaim within Fire, I felt  
I should respond to these points with my experience.


On Dec 7, 2006, at 6:35 PM, Evan Schoenberg wrote:

> Guess this is my Colin impression for the week... here goes a long  
> email:
>

Hey, it gets things done.  Also, I feel we would all prefer that  
everything is put out in the open initially so that the thread  
doesn't continue on forever with little tidbits leaked in here and  
there.

> About a year and a half ago, we began an experiment.  Could we  
> manhandle the Cocoa-Java bridge -- largely undocumented -- into  
> letting us utilize a promising Java library for AIM connectivity,  
> joscar?  joscar offered many great features which libgaim's AIM  
> implementation didn't.  Some choice selections:
> 1) Reliable AIM file transfer and direct connect

Most, if not nearly all, of this is already within Fire's libfaim  
code.  Much of it is written in Obj-C, which will make it harder to  
integrate into gaim's tree, but this was out of the convenience of  
sharing code between OFT (Oscar File Transfer) and ODC (Oscar Direct  
Connect).  It can be re-factored into non object oriented code  
without too much difficulty.  It could also use some cleanup in the  
process as well.

> 2) Client-side rate limiting management, which among other things  
> allows rapid retrieval of away messages at sign-on

I have considered actually implementing this in the past, but you  
seem to indicate below that it is there now.  There is one strange  
thought that nags at the back of my mind.  If this were implemented  
along with non-blocking IO (which I have implemented as well), then  
there is a buffer after the rate-limiting.  In certain network  
conditions, that buffer can negate the rate-limiting.  I do have to  
admit, this is a minor issue, especially since joscar has the same  
kind of non-blocking IO inside it.

> 3) AIM file transfer resume and folder transfer

Implemented within Fire's libfaim code.  I believe it is actually  
complete as well.  I actually wrote this code about 4 years ago, and  
so I am sure that it is completely incompatible with gaim's current  
file transfer interface.  The hooks shouldn't be too hard to change.

> 4) Trillan SecureIM compatibility

Can't answer to this one.  I only partially looked at it.  Since Fire  
had GPG, and then later OTR, I just didn't see the point.  In terms  
of implementation, I remember thinking it should be a plugin, like  
OTR, but have extra "meta-data" support by the library to work with  
it.  I would have to look at how secureIM is implemented again in  
order to expound on this idea.

Wasn't there something about supporting AIM's encryption for IMs in  
there as well?

> 5) AIM GetFile support

This is not hard to implement in the least.  First start a OFT  
connection in almost identical means to above.  Then, send a simple  
command which requests the file listing.  Then requesting a file is  
another command.  After that, the actual file sending is identical to  
file transfer listed above.  Just be aware, the file listing is a  
text file with constant column widths.  I think it limited the  
filesize to something like 7 or 8 digits, in decimal, so no big files  
here.

>
> We knew there were trade-offs from the get-go: without JNI or some  
> sort of java-to-byte-code compiler, the Java Vritual Machine has to  
> be running, which means increased memory usage, potentially  
> increased CPU usage, and a startup delay.  The state of libgaim's  
> AIM implementation was such that these seemed well worth it -- file  
> transfer and direct connect through varying network conditions  
> (anything but on the same LAN, basically) were effectively useless,  
> and everything else joscar offered was very shiny.
>
> Based on our decision to switch to joscar, we decided that there  
> was no reason not to embrace other Java solutions, and Andreas,  
> with Augie, and Alvaro worked on the Smack library throughout this  
> past summer.  Alvaro's efforts can be seen in Smack itself -- only  
> in the last weeks of the summer did he make changes within Adium,  
> and those weren't specific to Smack.  Andreas, on the other hand,  
> basically wrote a Jabber library based around Smack from the ground  
> up.
>
> I'm afraid we made a mistake and that it is better to correct it  
> now than to wait longer.
>
> The Cocoa-Java bridge is more than just 'largely undocumented', it  
> turns out -- it's deprecated, which means it definitely won't work  
> several years from now, and it's broken, which means it isn't  
> particularly reliable right now.  It's broken in 10.5 developer  
> builds, and there's no guarantee Apple will fix it even for 10.5,  
> and it's broken in 10.4 and 10.3 in that it crashes oddly and  
> randomly, with stack traces giving no indication of the source of  
> the problem or with a simple console message related to memory  
> management and an abort.
>

This is a sad situation to see.  Using java libraries opens up a lot  
of possibilities.  Personally I like the design of joscar far greater  
than that of libgaim (or more precisely libfaim).  Regardless of how  
superior the design, it is not worth keeping if the cost is continual  
use of a deprecated and broken interface.

> There are other options, theoretically, than the Cocoa-Java  
> Bridge.  Using JNI would be possible, most likely, but would be a  
> huge time investment with questionable results in terms of  
> performance.  The gjc compiler could potentially compile the  
> libraries, but it's not Java-library feature complete yet and  
> doesn't even compile on Darwin so far as I've been able to tell.
>

JNI is a pain, and still has the same issues with the RAM and CPU  
usage.  Personally I think that dealing with the breaking Cocoa-Java  
bridge would be easier than JNI.  If gjc were viable, I bet that  
would be the route to go, but I fear that it will not happen soon  
enough.

> We don't have the resources to continue to fight the protocol  
> fight, and we can't 'win' in any case when the enemy is an Apple  
> decision not to continue support for the bridge.  In addition,  
> speaking of resources, the RAM requirements of running the JavaVM  
> knock us squarely out of the 'lightweight' category and into the  
> 'heavyweight' -- something which I'd really like to avoid. We've  
> wrestled all the various Java problems back and forth... but it's a  
> losing battle.
>
> It's a losing battle, and the advantages of switching to other  
> libraries have grown smaller in the meantime.  Mark Doliner has  
> refactored and reorganized the libgaim OSCAR prpl -- largely  
> because of the impetus to change from us saying, "this is unusable,  
> and we need to switch to something better."  It is now clean enough  
> to support future development.  Its AIM file transfer and direct  
> connect are quite reliable now.  In the past week and a half, he  
> implemented rate limiting management, using an algorithm from the  
> joscar docs.  When it comes time for voice-video -- hopefully not  
> too far in the future -- we will, I believe, find that with these  
> cleanups and improvements it is very feasible to integrate the work  
> Alan has already done on Fire's libfaim-based AIM implementation  
> into libgaim.
>
> I point that out that the joscar docs were integral to libgaim's  
> implementation of the rate limiting logic because I don't want this  
> to be a library fight -- I'm proud that our work with joscar has  
> helped it improve and that other projects have picked it up for  
> use.  We're all on the same team.  There's still a ton to be  
> learned from Keith Lea's fantastic work with joscar, and a future  
> goal will definitely be to bring its other features into gaim's  
> implementation.  The gaim folk are very willing to work toward the  
> goal of a stable, feature-complete library intended for use in any  
> UI which wants it.
>

I don't want to see Adium in the library fight either.  The library  
fight is the reason why Eric left Fire development.

> ***
>
> I propose that we flip the #define back around to using libgaim for  
> AIM and .Mac, then build and release Adium 1.0b16.  adium-joscar  
> tickets which remain open in Trac will be marked invalid; closed  
> adium-joscar tickets on the 1.0 milestone (which should just be  
> ones which fixed Adium 0.8x bugs or added new features) should be  
> reevaluated and either reopened or marked closed on the AIM  
> component, as appropriate.  I've discussed this privately with  
> David and Chris and, at the risk of putting words in their mouths,  
> are in agreement -- at least with the general theme of this email  
> if not its full contents :)
>
> Not all work which has gone into joscar and Smack integration in  
> Adium is a loss, by any means.  In the process of implementing  
> each, many improvements were made to the Adium core.    Since the  
> libgaim Jabber library now supports the addition of external  
> plugins to add behaviors (for example, implementing the gmail mail  
> checking extension no longer requires modifications to the libgaim  
> jabber code itself), the higher-level work which was done for Smack  
> can probably be ported to working as a libgaim plugin as appropriate.
>
> Thoughts?
>
> -Evan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1603 bytes
Desc: not available
URL: <http://adium.im/pipermail/devel_adium.im/attachments/20061207/ef4254aa/attachment.p7s>