Loading images via http vs bytearray

Load images via http or using bytearray? which the best way?
It was long time since I wanted to perform this kind of test, but for different reasons I never had the time to do that.
Some days ago fnally I’ve found the time thanks to a project I’m involved into.
The problem.
A local webserver running flash remoting via pyamf and a flash standalone application which need to load tons of images locally.
I’ve noticed that sending from amf the list of images path to be loaded required too much time (for me), even if they’re loaded locally.
Because of this I decided to try a first benchmark loading a single image using the standard way: send the link from amf and then load it using the classic actionscript Loader.
The second test was to send directly from amf the image stream using a ByteArray and then loading the image in flash using the Loader.loadBytes method.
In this way I’ve noticed that the second task requires less time than the first one (more or less 40% less).
Unfortunately our application needs to load something like 50/100 images at the same time.
For this reason I did a new test loading 22 images of 500Kb each (10Mb total). For a better result I decided to use both pyamf and blazeds, to be sure there’s no problems in the language used. Moreover I used charles to register the benchmark of these tasks.
The results were unexpected!
First test: passing form amf the list of url images and loading using the standard flash Loader to load all of them simultaneously.
The time elapsed from the flash method call to the results was about 40 milliseconds. Then from the remote method result to the completion of all Loader about 1400 milliseconds.

Second test: passing from amf an arraycollection of bytearrays containing all the images stream to be loaded directly in flash.
The time elapsed form the flash method call to its result: about 2900 milliseconds. Time elapsed from the result to the all Loaders completion: about 900 milliseconds.
This results were unexpected for me, expecially because both with blazeds and pyamf sending 10Mb of bytearrays tooks something like 3 seconds!
At a first impression I thought the problem was the time for java and python to create the amf stream data, but after a deeper test I discovered that they took more or less 30ms to generate the amf stream and the real bottleneck was the http transfer of this stream data.

I’m attaching here the screenshots of the charles sessions using blazeds:

Here you can find also the complete benchmark result using flash trace within the swf:
————————–
BlazeDS
————————–
Method Elapsed-Time
call.test1 0
result.test1 40
complete.test1 1453
TEST1 TOTAL TIME: 1493
call.test2 0
result.test2 2919
complete.test2 921
TEST2 TOTAL TIME: 3840
————————–
PyAMF
————————–
call.test1 0
result.test1 32
complete.test1 953
TEST1 TOTAL TIME: 1049
call.test2 0
result.test2 2805
complete.test2 908
TEST2 TOTAL TIME: 4763
Here you can see the source code used for these tests:
In conclusion. While sending 10Mb of data ( for example bytearray ) requires more or less 2.5 seconds using flash remoting as single requests ( because the transfer rate of the webserver it’s about 4Mb/sec ), loading simultaneusly 20 images from the webserver, using http, tooks 1 second.
This is because the webserver for each requests opens a different thread to dispatch the request and in this way the total time to perform this task is less than the first method.

Flashlite and python games for my Nokia

Last week I bought my new mobile and I got a Nokia N73 phone.
After the first days playing with themes and ring tones, as usual with new toys :), I installed Flashlite 2.1! I must admit I never used and seen it before… so I decided to make a first quick experiment making a simple game.
Lucky that Adobe Device Central CS3 has just been released, so I could test the swf in different environments (even if I didnt understand how phones flashlite versions could be changed in the device central phone list..).

Anyway, finally I made this simple tetris remake with flashlite 2.1.
If you want to broke your phone you can try install it from here:
[attachments size=small docid=989 fields=none force_saveas=”1″ logged_users=”0″]

I also followed some tuts about creating .sis installer (for symbian phones) from here and also trying using other tolls such as SWF2GO but unfortunately I wan’t able to do that 🙁

Python… oh python 🙂
Then I discovered that there is a python version for series 60 (PyS60). I couldn’t believe to my eyes when I opened the Python shell from the phone menu! LOL

So, why not trying to make something also with python?
After installing tons and tons of software from Nokia, tutorials and example files I finally managed to start creating simple apps. Also this time I started with a simple game (my second fav arcade game): Asteroids.
This time I also managed to create the .sis file, even if not with the suggested process with py2sis, but using only Ensymble with this command:
ensymble_python2.5-0.22.py py2sis asteroidz.py asteroidz.sis --uid=0xA0002E1F --appname=asteroidz --caps="NONE" --lang=EN --shortcaption="asteroidz" --cert=mycert.cer --privkey=mykey.key --passphrase=password --verbose --version=1.0.1
You can download the .sis file and the source directly here (it requires python series 60 1.3.20 already installed):

[attachments docid=990 force_saveas=”1″ logged_users=”0″]

Anyway, please note that this is incomplete, it’s merely a test just for understand the python graphical api from Nokia.

I just discovered a new world 🙂

Actionscript parsing experiences: PyBison & PLY

My experiments with text-parsing continue..

PyBison
Last day I founded a python library (pybison) which runs the generated python parser at near the speed of C-based parsers, due to direct hooks into bison-generated C code.
Cool, unfortunately I couldn’t compiled it for Windows and so I made my test on Ubuntu only. What I did was just to export the already written lexer/grammar using bison2py (boundled with pybison) and run it.

If you want to take a look at the python parser try it by downloading the source code here.
The run.py file accept these parameters:
usage: run.py [options]
options:
-h, --help show this help message and exit
-v, --verbose Turn on verbose mode
-o OUTPUT_FILE, --output=OUTPUT_FILE
Don't print to stdout but to the output file
-i INPUT_FILE, --input=INPUT_FILE
Input file to be parsed
-x, --to-xml Returns a human-readable xml representation of the
parse tree
-b, --bison-debug Print the Bison/Flex debug

You can download the source code here

PLY
The second test I did was using PLY, an implementation of lex and yacc parsing tools for Python. Being implemented entirely in python it should be much more slower that pybison, but I didn’t find any difference with the pybison parser version. In fact PLY , like the traditional bison, creates tables starting from the grammar syntax.
Ok, Both of the implementations are slower that the pure C parser, but extremely faster that antlr!
(They took more or less 0.02 to 0.5 secs for parsing and generating the AST.)
Unlike pybison PLY is still mantained and offers more features and a better error handling.. even if the whole grammar has to be rewritten in python, and it can be compiled in Windows too.

To run the test just write:
python run.py {filename}
P.S. Unfortunately the yacc parser isn’t yet complete because I still need to find a way for parsing correctly E4X and XML syntax..

ActionScript Parsing, the YACC revenge :)

After my first attempts with ANTLR scanners in python/java I decided to start back with Bison/Flex again to see the difference in performances.
So first I need to wrote from scratch the grammar/lexer files using only the ECMAScript 4 specifications and much patience (the elastic grammar file help me a lot too).

After finishing a first version of the parser I tested it on the same file (75Kb actionscript file) which both java and python parsed in more than 1 second.
The result was unbelievable: 0.02 seconds for that file!

Then I tested it on multiple files, and for about 320 files of the whole adobe corelib library it took 220ms

Ok, the parser it’s not yet complete and doesn’t care about regexp and xml syntax, but its performance convinced me enough…
Now, the next step is to finish and test the parser and finally create a python library using pyrex, then a benchmark test again.

If someone is interested in testing the parser, download it (use “parser –help” form the command line for usage help), but remember this is only a first test.. not really helpful right now (I just wanted to share my text/parsing experiences).

An experience with antlr, java and python

I just wanted to share a little experience with generating an AS3 parser using antlr and python.
I was trying first to create the parser using GNU Flex and Bison in C, probably the best way for a very performancing parser.
Yeah, that’s right.. but looking at the antlr syntax I realized that’s easier and easier.
Moreover I start using this very useful eclipse plugin for antlr debugging which made my life easier!

The grammar file I created is a compromise between the asdt grammar file and the ECMA-262 grammar specification.

Once finished working on my eclipse project I’ve managed to parse succesfully all the adobe corelibs files using this java test file:
package org.sepy.core.parsers.as3;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import antlr.CommonAST;
import antlr.RecognitionException;
import antlr.TokenStreamException;
public class Application {
public static void main(String argv[])
{
if(argv.length > 0)
{
File file = new File(argv[0]);
if(file.exists())
{
FileInputStream is = null;
try {
is = new FileInputStream(file);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
AS3Lexer L = new AS3Lexer(is);
AS3Parser P = new AS3Parser(L);
try {
P.compilationUnit();
} catch (RecognitionException e) {
// TODO Auto-generated catch block
System.out.println(" line=" + e.line + ", column="+ e.column);
System.out.println(e.getMessage());
e.printStackTrace(System.err);
} catch (TokenStreamException e) {
// TODO Auto-generated catch block
System.out.println(" line=" + L.getLine() + ", column="+ L.getColumn());
System.out.println(L.getGuessInfo());
System.out.println(e.getMessage());
e.printStackTrace(System.err);
}
CommonAST.setVerboseStringConversion(false, P.getTokenNames());
CommonAST ast = (CommonAST) P.getAST();
System.out.println("Tree:");
System.out.println(ast.toStringTree());
}
}
}
}

Ok, done that I decided to export the grammar file for python (thanks to antrl python export feature).
Everything works fine also for python, but I realized that the python script were so much slower than the java one!
import sys
import antlr
import AS3Parser
import AS3Lexer
L = AS3Lexer.Lexer(filename);
P = AS3Parser.Parser(L);
P.setFilename(filename)
try:
P.compilationUnit();
ast = P.getAST();
except:
pass

On a 75Kb actionscript file the python script took about 7 seconds to run, while the java application only 2 seconds. I know python interpreter caould be slower than many other languages, but I never thought so much slower.
So I run the python hotshot profiler to see which could be the bottleneck in the python script and I found most of the problems were due to unuseless antlr (the python module) method’s calls.
After making corrections to the antlr.py file the same script took exactly half of the time. Now 3 seconds. Wow 🙂
But not fast enough.
So I enabled for the antlr python script psyco module and this time the same script took only 1.6 seconds.
Now the python script is fast enough, even if I’m sure I can make more optimizations in the antlr module…

Red5 with javascript python ruby and coffee?

Red5 (the opensource flash multimedia server) will support server side javascript, Ruby, Python and Java?
That’s terrific!
Well, it seems the next 0.6 RC1 version of Red5 will do that! The release is scheduled by next Monday and if that’s true I definitively will find the time to look inside it 🙂