i found a bottleneck

Discussion:

ricardo lafuente

2009-06-20 17:51:19 UTC

hey guys,

after the file structure refactoring, i couldn't help taking another
stab at finding the bottlenecks in the transform code.
it turns out that after
- removing individual path transforms, which breaks stuff like
rect(0,0,10,10, scale=(2,2)), but that won't be hard to put back again
and is a bit pointless anyway
- having the canvas directly process the drawables as Bot passes them,
instead of building a drawables stack that get drawn in the end of the
loop; then it applies the Bot transform matrix instead of the one in the
drawable
- and doing some cleanup on the canvas drawing code,

stuff looks quite a bit faster to me. My eee shows something like a 2x
increase in speed in the planets script that Stu sent to the list some
months ago.
if you feel like trying the new code, get it through

hg clone http://bitbucket.org/rlafuente/shoebot-filestructure

the script i mentioned is in examples/basic/transform/planets.bot (in
the main branch it's in examples/nodebox/transforms_stress_test.bot IIRC)

can any of you guys test it to make sure i'm not seeing things?

:r

architetto francesco fantoni <hva - hermanitos verdes architetti>

2009-06-20 18:23:51 UTC

Permalink

hi ricardo,
seems a lot faster indeed.
there are some things still broken anyway, two I noticed are:

-text is not working
- .fill and .stroke attributes for objects seems not to be working
anymore either. (e.g.sun_planet_moon_2.bot in nodebox examples
subfolder)

francesco

Post by ricardo lafuente
hey guys,
after the file structure refactoring, i couldn't help taking another
stab at finding the bottlenecks in the transform code.
it turns out that after
- removing individual path transforms, which breaks stuff like
rect(0,0,10,10, scale=(2,2)), but that won't be hard to put back again
and is a bit pointless anyway
- having the canvas directly process the drawables as Bot passes them,
instead of building a drawables stack that get drawn in the end of the
loop; then it applies the Bot transform matrix instead of the one in the
drawable
- and doing some cleanup on the canvas drawing code,
stuff looks quite a bit faster to me. My eee shows something like a 2x
increase in speed in the planets script that Stu sent to the list some
months ago.
if you feel like trying the new code, get it through
hg clone http://bitbucket.org/rlafuente/shoebot-filestructure
the script i mentioned is in examples/basic/transform/planets.bot (in
the main branch it's in examples/nodebox/transforms_stress_test.bot IIRC)
can any of you guys test it to make sure i'm not seeing things?
:r
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net

architetto francesco fantoni <hva - hermanitos verdes architetti>

2009-06-20 18:31:59 UTC

Permalink

colors library is broken as well with file-structure refactoring:
it used to do things like "from shoebot import Color as BaseColor" or
"from shoebot import RGB", and now it doesn't know where to find the
classes it needs

Il giorno sab, 20/06/2009 alle 20.23 +0200, architetto francesco fantoni

Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
hi ricardo,
seems a lot faster indeed.
-text is not working
- .fill and .stroke attributes for objects seems not to be working
anymore either. (e.g.sun_planet_moon_2.bot in nodebox examples
subfolder)
francesco

_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net

architetto francesco fantoni <hva - hermanitos verdes architetti>

2009-06-20 18:42:24 UTC

Permalink

clipping paths seem not to work anymore either

Il giorno sab, 20/06/2009 alle 20.31 +0200, architetto francesco fantoni

Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
it used to do things like "from shoebot import Color as BaseColor" or
"from shoebot import RGB", and now it doesn't know where to find the
classes it needs
Il giorno sab, 20/06/2009 alle 20.23 +0200, architetto francesco fantoni

_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net

ricardo lafuente

2009-06-24 19:16:18 UTC

Permalink

clipping paths, images and text are now all working in the new branch. Whew.
i renamed the 'RestoreCtx' class to 'EndClip', which makes its purpose
more evident.
check out the commit messages to see what other things i've tweaked
there ().
while trying to fix all the broken stuff i ran into a few issues and
fixed most of them.

Francesco, you've got my total respect for the great hacks you made in
clips and text -- took me a while to get it, but everything is really
neatly done. I've just repositioned some calls, but wow, great work. I
still don't know how you got the clips to work with save/restore, Cairo
just drove me mad and i ended up going back to your double-save hack,
which i still don't know how works (i just know that it does :)

one thing on my mind is to review the ShoebotDrawingArea so that expose
events don't run the script again (only scripts with a draw loop should
be animated/refreshed, but you'll see them being re-run if you resize
the window. They should just redraw the canvas, not run the script again.)

oh, and i got a Drawbot implementation almost there, just missing text
support. Woo! I'm a happy hacker tonight :)

please test some stuff if you feel like it. I'd love to be confident
enough to pull all these changes by the end of the week. Francesco, let
me know if you need help updating the lib imports.
cheers y'all

Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
clipping paths seem not to work anymore either
Il giorno sab, 20/06/2009 alle 20.31 +0200, architetto francesco fantoni

_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net

architetto francesco fantoni <hva - hermanitos verdes architetti>

2009-06-24 19:53:37 UTC

Permalink

wow ricardo! such a great amount of hacking. Chapeau!
I tested it a bit and it seems to work really fine, and fast.
I only found a little problem in scripts with a draw() function when
size() statement is outside the setup() function. In that case the
window is created with default size and not according to size()
statement. (actually in this case the script itself is sort of wrong, as
it makes really no sense to leave the size() call outside of setup)

Thanks for you kind words of appreciation for my little hacks on text
and clip paths. The funniest thing about it is I don't know how things
work either: i completely forgot about it! (I'm getting old, it seems)
;)
I'll do some more testing.
I didn't had the time to fix libraries yet, sorry...

F.

Post by ricardo lafuente
clipping paths, images and text are now all working in the new branch. Whew.
i renamed the 'RestoreCtx' class to 'EndClip', which makes its purpose
more evident.
check out the commit messages to see what other things i've tweaked
there ().
while trying to fix all the broken stuff i ran into a few issues and
fixed most of them.
Francesco, you've got my total respect for the great hacks you made in
clips and text -- took me a while to get it, but everything is really
neatly done. I've just repositioned some calls, but wow, great work. I
still don't know how you got the clips to work with save/restore,
Cairo just drove me mad and i ended up going back to your double-save
hack, which i still don't know how works (i just know that it does :)
one thing on my mind is to review the ShoebotDrawingArea so that
expose events don't run the script again (only scripts with a draw
loop should be animated/refreshed, but you'll see them being re-run if
you resize the window. They should just redraw the canvas, not run the
script again.)
oh, and i got a Drawbot implementation almost there, just missing text
support. Woo! I'm a happy hacker tonight :)
please test some stuff if you feel like it. I'd love to be confident
enough to pull all these changes by the end of the week. Francesco,
let me know if you need help updating the lib imports.
cheers y'all

Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
clipping paths seem not to work anymore either
Il giorno sab, 20/06/2009 alle 20.31 +0200, architetto francesco fantoni

_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net

ricardo lafuente

2009-06-25 15:20:13 UTC

Permalink

Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
wow ricardo! such a great amount of hacking. Chapeau!
I tested it a bit and it seems to work really fine, and fast.
I only found a little problem in scripts with a draw() function when
size() statement is outside the setup() function. In that case the
window is created with default size and not according to size()
statement. (actually in this case the script itself is sort of wrong, as
it makes really no sense to leave the size() call outside of setup)

i think i sorted this one out, moved size stuff to the Canvas and it now
appears to work. Today i noticed that we have some stuff spread between
Bot and Canvas (wrt to defining current state). I'm thinking that later,
all the state stuff (current fill/stroke, current transform, etc.)
should be placed inside Canvas; Bot should only translate the script
language (Drawbot/Nodebox/...) into canvas drawing commands, not store
values. ATM Canvas needs to refer to the Bot to get some of these
states, i think this should not happen. And i think we'd get some good
performance gains as well.

anyhow, this might be left for later, too many upheavals already for
this week :o)

oh and btw, i'm working on Gedit syntax highlighting for Shoebot
functions; see Loading Image...

for
a screenshot of Gedit running a Drawbot script (yeah, Drawbot :)

Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
Thanks for you kind words of appreciation for my little hacks on text
and clip paths. The funniest thing about it is I don't know how things
work either: i completely forgot about it! (I'm getting old, it seems)
;)
I'll do some more testing.
I didn't had the time to fix libraries yet, sorry...
F.

Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
clipping paths seem not to work anymore either
Il giorno sab, 20/06/2009 alle 20.31 +0200, architetto francesco fantoni

_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net

Dave Crossland

2009-06-25 15:29:01 UTC

Permalink

btw, i'm working on Gedit syntax highlighting for Shoebot functions;
see http://ricardo.koizo.org/files/drawbot-screenshot.png for a screenshot
of Gedit running a Drawbot script (yeah, Drawbot :)

Very nice!

ricardo lafuente

2009-06-25 15:58:04 UTC

Permalink

Post by ricardo lafuente

i think i sorted this one out, moved size stuff to the Canvas and it
now appears to work.

dang, too early to party. size is not there yet, and since i had to
reintroduce TransformMixins to deal with paths that are called with
draw=False, we got slowness again :/ i'll have to spend some more time
on this

ricardo lafuente

2009-06-20 19:03:36 UTC

Permalink

ahh, had to do with me creating a colors.py file. Renamed it to
basecolor.py. Updated the branch repo
i'll look into the lib tomorrow to check what names need to be changed,
as data structures live now in shoebot.data .

thank you for your swift eyes ;)

Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
Il giorno sab, 20/06/2009 alle 20.23 +0200, architetto francesco fantoni

_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net

ricardo lafuente

2009-06-20 18:52:18 UTC

Permalink

Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
hi ricardo,
seems a lot faster indeed.
-text is not working

this probably has to do with the Text class working with the current
CairoCanvas context. I've been fiddling with the canvas a lot, so maybe
making Text independent of it will make things better.
i've fixed it in a way, but transforms and positioning are not applied
apparently. I'll look into this.

Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
- .fill and .stroke attributes for objects seems not to be working
anymore either. (e.g.sun_planet_moon_2.bot in nodebox examples
subfolder)

well spotted, i'll also try and find what went wrong there...

Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
francesco

_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net

ricardo lafuente

2009-06-21 06:53:47 UTC

Permalink

this raises a rather important design issue.
that script does something on the lines of:

r = rect(0,0,10,10)
r.fill(1,0,0)
r.stroke(0,0,1)

the Nodebox way of dealing with drawables is to store them as objects
and only pass them to the canvas at the end of the loop. That way, their
properties can still be accessed and changed before drawing.

this contrasts with the true finite-state machine model used by Cairo,
Processing and Drawbot (not sure about the last one), where the
drawables are not stored but instead passed directly to the canvas as
they are called.
i switched to this new model in the new branch i've been hacking on
(which would be the basis for Shoebot 0.3), as keeping the objects
stored until the end of the loop causes a big memory and CPU overhead
that became evident on the animation slowdowns.

under the Cairo model, the behaviour of the snippet above is easily
achievable:

fill(1,0,0)
stroke(0,0,1)
rect(0,0,10,10)

and this also works:

r = rect(0,0,10,10, draw=False)
r.fill(1,0,0)
r.stroke(0,0,1)
drawpath(r)

what does not work is setting drawable attributes after they are drawn,
since they're sent to the canvas and forgotten.

i am reminded of the Python mantra "There should be one-- and preferably
only one --obvious way to do it". And to be honest, i would rather stick
to the Cairo way of doing things rather than having major slowdowns in
order to support a Nodebox feature that is IMHO bad design, not to
mention superfluous and redundant -- it can be done in at least two
other ways that require at most a couple more lines of code.

so this introduces an issue of minor backwards incompatibility. My
proposal would be to raise an error when trying to set attributes of
drawables that were already sent to the canvas (i.e. created without
draw=False), and port the existing example scripts that do this (the
sun_planet_moon example is the only one i'm aware of that does it).

again, this would be a design change that wouldn't affect 99% of use
cases. For the remaining 1% it just needs a couple of lines more. And
IMO it's really worth it, performance- and simplicity-wise.

what's your thoughts? I might be missing some cases where this feature
would be required.

Stuart Axon

2009-06-21 01:51:29 UTC

Permalink

This speedup looks great :)

- Heres to finding many more :-D !

----- Original Message ----
From: ricardo lafuente <***@sollec.org>
To: shoebot mailing list <shoebot-devel-***@public.gmane.org>
Sent: Saturday, June 20, 2009 6:51:19 PM
Subject: [shoebot-devel] i found a bottleneck

hey guys,

after the file structure refactoring, i couldn't help taking another stab at finding the bottlenecks in the transform code.
it turns out that after
- removing individual path transforms, which breaks stuff like rect(0,0,10,10, scale=(2,2)), but that won't be hard to put back again and is a bit pointless anyway
- having the canvas directly process the drawables as Bot passes them, instead of building a drawables stack that get drawn in the end of the loop; then it applies the Bot transform matrix instead of the one in the drawable
- and doing some cleanup on the canvas drawing code,

stuff looks quite a bit faster to me. My eee shows something like a 2x increase in speed in the planets script that Stu sent to the list some months ago.
if you feel like trying the new code, get it through

hg clone http://bitbucket.org/rlafuente/shoebot-filestructure

the script i mentioned is in examples/basic/transform/planets.bot (in the main branch it's in examples/nodebox/transforms_stress_test.bot IIRC)

can any of you guys test it to make sure i'm not seeing things?

:r

Continue reading on narkive:

Search results for 'i found a bottleneck' (Questions and Answers)

replies

am i bottlenecking with this computer?

started 2011-10-02 07:12:03 UTC

desktops

replies

how do i fix a 'bottleneck'?

started 2013-08-18 02:58:07 UTC

hardware

replies

Am I CPU bottlenecking?