Discussion:
why Shoebot can be slow, and a possible way out
ricardo lafuente
2009-05-10 17:03:36 UTC
Permalink
hey all,

i was reading through data.py and was reminded of Stu's profiling
results with gtk_main_iteration.

I believe our main problem is transforms; for each path that a user
creates, Shoebot has to find its centerpoint to determine the transform.
The only way right now is using Cairo, since we have no other lib to
deal with this, AFAIK. This really sucks, using Cairo for such a simple
operation.

not only that, we have to do a lot of matrix parsing and re-generating
because there is no native Python lib for handling vector shapes,
matrices and data. Our implementation is a bit hackish -- nevertheless,
what we have works and looks great (and supports both Cairo's corner
transform mode as well as Nodebox's center mode), but the footprint of
all these operations begins to show when we use stacked transforms.

again, our solution would be a library to handle the Path, Matrix, and
all the primitive object types, which we could then use as a replacement
for most of the stuff in data.py. I've looked around a lot and couldn't
find anything similar in Python, but there's still hope -- lib2geom, the
C++ lib that powers Inkscape's path operations, already has some
bare-bones Python bindings.

i've talked to them on their Jabber channel and learned a bit how i
could help (njh is a great and patient guy :) -- it's mostly an effort
of reading the already existing bindings and writing the same for the
remaining ones. I've sent a patch already for binding a couple of
objects but there's still some work to be done.

the advantage of this would be, well, awesome: lib2geom supports boolean
operations between paths; for comparison, Nodebox's bezier library is
able to do that, albeit resulting in paths made out of only straight
lines, no curves. And well, if Inkscape uses it as a back-end, we can be
sure it will be a good bet.

i will be going on trying to add to their binding effort, but my almost
non-existent knowledge of C++ makes it tough on me; if there's anyone
willing to lend a hand to this, well, it would be really really great to
have a Python way to deal with beziers and related stuff, all this
besides how useful it would be for Shoebot.

if anyone's up for a sprint of sorts to work on this, it would be tops :-)

:r
architetto francesco fantoni <hva - hermanitos verdes architetti>
2009-05-10 18:06:31 UTC
Permalink
Me too I thought that the main bottleneck is in the transforms, I have
scripts with images, and text and clipping paths that can run very fast,
while simple scripts with lots of transformations are much slower.

I don't know much about lib2geom, i compiled it and had a look at the
examples, but i was scared by the total lack of documentation.
How about matrix handling using something like numpy? I don't know if it
could be done, nor if it could be faster than cairo or not...
Post by ricardo lafuente
hey all,
i was reading through data.py and was reminded of Stu's profiling
results with gtk_main_iteration.
I believe our main problem is transforms; for each path that a user
creates, Shoebot has to find its centerpoint to determine the transform.
The only way right now is using Cairo, since we have no other lib to
deal with this, AFAIK. This really sucks, using Cairo for such a simple
operation.
not only that, we have to do a lot of matrix parsing and re-generating
because there is no native Python lib for handling vector shapes,
matrices and data. Our implementation is a bit hackish -- nevertheless,
what we have works and looks great (and supports both Cairo's corner
transform mode as well as Nodebox's center mode), but the footprint of
all these operations begins to show when we use stacked transforms.
again, our solution would be a library to handle the Path, Matrix, and
all the primitive object types, which we could then use as a replacement
for most of the stuff in data.py. I've looked around a lot and couldn't
find anything similar in Python, but there's still hope -- lib2geom, the
C++ lib that powers Inkscape's path operations, already has some
bare-bones Python bindings.
i've talked to them on their Jabber channel and learned a bit how i
could help (njh is a great and patient guy :) -- it's mostly an effort
of reading the already existing bindings and writing the same for the
remaining ones. I've sent a patch already for binding a couple of
objects but there's still some work to be done.
the advantage of this would be, well, awesome: lib2geom supports boolean
operations between paths; for comparison, Nodebox's bezier library is
able to do that, albeit resulting in paths made out of only straight
lines, no curves. And well, if Inkscape uses it as a back-end, we can be
sure it will be a good bet.
i will be going on trying to add to their binding effort, but my almost
non-existent knowledge of C++ makes it tough on me; if there's anyone
willing to lend a hand to this, well, it would be really really great to
have a Python way to deal with beziers and related stuff, all this
besides how useful it would be for Shoebot.
if anyone's up for a sprint of sorts to work on this, it would be tops :-)
:r
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
architetto francesco fantoni <hva - hermanitos verdes architetti>
2009-05-10 18:28:03 UTC
Permalink
But you're right Ricardo, lib2geom is the way to go.

Il giorno dom, 10/05/2009 alle 20.06 +0200, architetto francesco fantoni
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
Me too I thought that the main bottleneck is in the transforms, I have
scripts with images, and text and clipping paths that can run very fast,
while simple scripts with lots of transformations are much slower.
I don't know much about lib2geom, i compiled it and had a look at the
examples, but i was scared by the total lack of documentation.
How about matrix handling using something like numpy? I don't know if it
could be done, nor if it could be faster than cairo or not...
Post by ricardo lafuente
hey all,
i was reading through data.py and was reminded of Stu's profiling
results with gtk_main_iteration.
I believe our main problem is transforms; for each path that a user
creates, Shoebot has to find its centerpoint to determine the transform.
The only way right now is using Cairo, since we have no other lib to
deal with this, AFAIK. This really sucks, using Cairo for such a simple
operation.
not only that, we have to do a lot of matrix parsing and re-generating
because there is no native Python lib for handling vector shapes,
matrices and data. Our implementation is a bit hackish -- nevertheless,
what we have works and looks great (and supports both Cairo's corner
transform mode as well as Nodebox's center mode), but the footprint of
all these operations begins to show when we use stacked transforms.
again, our solution would be a library to handle the Path, Matrix, and
all the primitive object types, which we could then use as a replacement
for most of the stuff in data.py. I've looked around a lot and couldn't
find anything similar in Python, but there's still hope -- lib2geom, the
C++ lib that powers Inkscape's path operations, already has some
bare-bones Python bindings.
i've talked to them on their Jabber channel and learned a bit how i
could help (njh is a great and patient guy :) -- it's mostly an effort
of reading the already existing bindings and writing the same for the
remaining ones. I've sent a patch already for binding a couple of
objects but there's still some work to be done.
the advantage of this would be, well, awesome: lib2geom supports boolean
operations between paths; for comparison, Nodebox's bezier library is
able to do that, albeit resulting in paths made out of only straight
lines, no curves. And well, if Inkscape uses it as a back-end, we can be
sure it will be a good bet.
i will be going on trying to add to their binding effort, but my almost
non-existent knowledge of C++ makes it tough on me; if there's anyone
willing to lend a hand to this, well, it would be really really great to
have a Python way to deal with beziers and related stuff, all this
besides how useful it would be for Shoebot.
if anyone's up for a sprint of sorts to work on this, it would be tops :-)
:r
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
ricardo lafuente
2009-05-10 21:34:09 UTC
Permalink
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
I don't know much about lib2geom, i compiled it and had a look at the
examples, but i was scared by the total lack of documentation.
How about matrix handling using something like numpy? I don't know if it
could be done, nor if it could be faster than cairo or not...
True, it could definitely have some helper docs around. But since it's a
rather low-level lib, maybe that's not their priority. The examples are
pretty neat and run on Cairo -- try the python ones, they're practically
self-documenting. There's also something on 2geom on the Inkscape wiki
that briefly shows how it works (writing this off-line, try googling).

Something like Numpy would go better than Cairo, no doubt. I do remember
being pretty baffled by the complexity of the numpy concepts and quickly
got lost. But hey, you did work with it once, do you think it's doable?
(though i totally understand you won't want to place your eyes on an
affine matrix ever again :o)

On the other hand, 2geom have their own matrix implementation, and it's
already up and running on the Python bindings. Right now, they're just
missing bezier paths and other less simple primitives (like ellipses).
I'm personally very impressed by the whole of 2geom, especially after
playing with the existing python bindings on an ipython session. Not to
mention that Inkscape, which fundamentally relies on 2geom, regularly
contributes with patches.

Pardon my excitement, but i really think it's the missing piece of the
whole puzzle. Having a lib to take care of the abstract object data
(beziers, points, etc.) would fit everything together: Bot reads the
user scripts, 2geom objects are created, and finally passed to the
canvas, which will then call the appropriate commands on the back-end of
choice. And it would do away with our current mess regarding transforms
(a mess that works nonetheless, i'd emphasise :)

So yes, maybe it's not the kind of effort a hacker would fancy, but i
was hopeful i'd find someone who might be able to lend a hand.
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
Post by ricardo lafuente
hey all,
i was reading through data.py and was reminded of Stu's profiling
results with gtk_main_iteration.
I believe our main problem is transforms; for each path that a user
creates, Shoebot has to find its centerpoint to determine the transform.
The only way right now is using Cairo, since we have no other lib to
deal with this, AFAIK. This really sucks, using Cairo for such a simple
operation.
not only that, we have to do a lot of matrix parsing and re-generating
because there is no native Python lib for handling vector shapes,
matrices and data. Our implementation is a bit hackish -- nevertheless,
what we have works and looks great (and supports both Cairo's corner
transform mode as well as Nodebox's center mode), but the footprint of
all these operations begins to show when we use stacked transforms.
again, our solution would be a library to handle the Path, Matrix, and
all the primitive object types, which we could then use as a replacement
for most of the stuff in data.py. I've looked around a lot and couldn't
find anything similar in Python, but there's still hope -- lib2geom, the
C++ lib that powers Inkscape's path operations, already has some
bare-bones Python bindings.
i've talked to them on their Jabber channel and learned a bit how i
could help (njh is a great and patient guy :) -- it's mostly an effort
of reading the already existing bindings and writing the same for the
remaining ones. I've sent a patch already for binding a couple of
objects but there's still some work to be done.
the advantage of this would be, well, awesome: lib2geom supports boolean
operations between paths; for comparison, Nodebox's bezier library is
able to do that, albeit resulting in paths made out of only straight
lines, no curves. And well, if Inkscape uses it as a back-end, we can be
sure it will be a good bet.
i will be going on trying to add to their binding effort, but my almost
non-existent knowledge of C++ makes it tough on me; if there's anyone
willing to lend a hand to this, well, it would be really really great to
have a Python way to deal with beziers and related stuff, all this
besides how useful it would be for Shoebot.
if anyone's up for a sprint of sorts to work on this, it would be tops :-)
:r
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
architetto francesco fantoni <hva - hermanitos verdes architetti>
2009-05-11 19:46:37 UTC
Permalink
Ricardo, I think you're absolutely right thinking that 2geom is perfect
for bezier handling, as you said it would add all sort of possibility
for boolean operations and mathematics on paths, and that would be
really great (and it seems rather powerful judging from inkscape).
What do you think about text and images? can be managed with lib2geom as
well, or we should have an alternative matrix handling for their
transformations?
I seriously doubt i could manage numpy well enough, i just scratched the
surface of it as i needed an array handling routine for the Image code
of shoebot and for video frames handling in sbopencv library, but I will
try to have a look at 2geom and numpy when I have the time, in order to
be able to possibly help when you decide to try the first implementation
of a new "next generation" shoebot ;)
ciao, francesco
Post by ricardo lafuente
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
I don't know much about lib2geom, i compiled it and had a look at the
examples, but i was scared by the total lack of documentation.
How about matrix handling using something like numpy? I don't know if it
could be done, nor if it could be faster than cairo or not...
True, it could definitely have some helper docs around. But since it's a
rather low-level lib, maybe that's not their priority. The examples are
pretty neat and run on Cairo -- try the python ones, they're practically
self-documenting. There's also something on 2geom on the Inkscape wiki
that briefly shows how it works (writing this off-line, try googling).
Something like Numpy would go better than Cairo, no doubt. I do remember
being pretty baffled by the complexity of the numpy concepts and quickly
got lost. But hey, you did work with it once, do you think it's doable?
(though i totally understand you won't want to place your eyes on an
affine matrix ever again :o)
On the other hand, 2geom have their own matrix implementation, and it's
already up and running on the Python bindings. Right now, they're just
missing bezier paths and other less simple primitives (like ellipses).
I'm personally very impressed by the whole of 2geom, especially after
playing with the existing python bindings on an ipython session. Not to
mention that Inkscape, which fundamentally relies on 2geom, regularly
contributes with patches.
Pardon my excitement, but i really think it's the missing piece of the
whole puzzle. Having a lib to take care of the abstract object data
(beziers, points, etc.) would fit everything together: Bot reads the
user scripts, 2geom objects are created, and finally passed to the
canvas, which will then call the appropriate commands on the back-end of
choice. And it would do away with our current mess regarding transforms
(a mess that works nonetheless, i'd emphasise :)
So yes, maybe it's not the kind of effort a hacker would fancy, but i
was hopeful i'd find someone who might be able to lend a hand.
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
Post by ricardo lafuente
hey all,
i was reading through data.py and was reminded of Stu's profiling
results with gtk_main_iteration.
I believe our main problem is transforms; for each path that a user
creates, Shoebot has to find its centerpoint to determine the transform.
The only way right now is using Cairo, since we have no other lib to
deal with this, AFAIK. This really sucks, using Cairo for such a simple
operation.
not only that, we have to do a lot of matrix parsing and re-generating
because there is no native Python lib for handling vector shapes,
matrices and data. Our implementation is a bit hackish -- nevertheless,
what we have works and looks great (and supports both Cairo's corner
transform mode as well as Nodebox's center mode), but the footprint of
all these operations begins to show when we use stacked transforms.
again, our solution would be a library to handle the Path, Matrix, and
all the primitive object types, which we could then use as a replacement
for most of the stuff in data.py. I've looked around a lot and couldn't
find anything similar in Python, but there's still hope -- lib2geom, the
C++ lib that powers Inkscape's path operations, already has some
bare-bones Python bindings.
i've talked to them on their Jabber channel and learned a bit how i
could help (njh is a great and patient guy :) -- it's mostly an effort
of reading the already existing bindings and writing the same for the
remaining ones. I've sent a patch already for binding a couple of
objects but there's still some work to be done.
the advantage of this would be, well, awesome: lib2geom supports boolean
operations between paths; for comparison, Nodebox's bezier library is
able to do that, albeit resulting in paths made out of only straight
lines, no curves. And well, if Inkscape uses it as a back-end, we can be
sure it will be a good bet.
i will be going on trying to add to their binding effort, but my almost
non-existent knowledge of C++ makes it tough on me; if there's anyone
willing to lend a hand to this, well, it would be really really great to
have a Python way to deal with beziers and related stuff, all this
besides how useful it would be for Shoebot.
if anyone's up for a sprint of sorts to work on this, it would be tops :-)
:r
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
Stuart Axon
2009-05-11 21:53:41 UTC
Permalink
I wasn't sure if the slowdown was
- GTK/Pygtk
- The transforms (or other things we do with cairo that may be inefficient)
- Cairo in general
(or some combination of them).

To try and rule out the first one of these I tried drawing onto an opengl surface using pyglet
http://bitbucket.org/stuaxo/shoebot-pyglet/

It seems a little faster, but not significantly (this is subjective as I've only tested one bot so
far)

If you want to test your bot you can check it out, change size to 600,600 as it doesn't respect
size.
(only tested with sbot).


My next step is to try and convert some bots to pure cairo and see if they run faster, I'd
say it seems like your on the right track with making the transforms and matrices quicker
though.

If it turns out to be cairo then hopefully I can push some bugs in their direction.



----- Original Message ----
From: "architetto francesco fantoni <hva - hermanitos verdes architetti>" <francesco-pB81QDORtWSUBmjqxMAjcGD2FQJk+8+***@public.gmane.org>
To: shoebot-devel-***@public.gmane.org
Sent: Monday, May 11, 2009 8:46:37 PM
Subject: Re: [shoebot-devel] why Shoebot can be slow, and a possible way out

Ricardo, I think you're absolutely right thinking that 2geom is perfect
for bezier handling, as you said it would add all sort of possibility
for boolean operations and mathematics on paths, and that would be
really great (and it seems rather powerful judging from inkscape).
What do you think about text and images? can be managed with lib2geom as
well, or we should have an alternative matrix handling for their
transformations?
I seriously doubt i could manage numpy well enough, i just scratched the
surface of it as i needed an array handling routine for the Image code
of shoebot and for video frames handling in sbopencv library, but I will
try to have a look at 2geom and numpy when I have the time, in order to
be able to possibly help when you decide to try the first implementation
of a new "next generation" shoebot ;)
ciao, francesco
Post by ricardo lafuente
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
I don't know much about lib2geom, i compiled it and had a look at the
examples, but i was scared by the total lack of documentation.
How about matrix handling using something like numpy? I don't know if it
could be done, nor if it could be faster than cairo or not...
True, it could definitely have some helper docs around. But since it's a
rather low-level lib, maybe that's not their priority. The examples are
pretty neat and run on Cairo -- try the python ones, they're practically
self-documenting. There's also something on 2geom on the Inkscape wiki
that briefly shows how it works (writing this off-line, try googling).
Something like Numpy would go better than Cairo, no doubt. I do remember
being pretty baffled by the complexity of the numpy concepts and quickly
got lost. But hey, you did work with it once, do you think it's doable?
(though i totally understand you won't want to place your eyes on an
affine matrix ever again :o)
On the other hand, 2geom have their own matrix implementation, and it's
already up and running on the Python bindings. Right now, they're just
missing bezier paths and other less simple primitives (like ellipses).
I'm personally very impressed by the whole of 2geom, especially after
playing with the existing python bindings on an ipython session. Not to
mention that Inkscape, which fundamentally relies on 2geom, regularly
contributes with patches.
Pardon my excitement, but i really think it's the missing piece of the
whole puzzle. Having a lib to take care of the abstract object data
(beziers, points, etc.) would fit everything together: Bot reads the
user scripts, 2geom objects are created, and finally passed to the
canvas, which will then call the appropriate commands on the back-end of
choice. And it would do away with our current mess regarding transforms
(a mess that works nonetheless, i'd emphasise :)
So yes, maybe it's not the kind of effort a hacker would fancy, but i
was hopeful i'd find someone who might be able to lend a hand.
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
Post by ricardo lafuente
hey all,
i was reading through data.py and was reminded of Stu's profiling
results with gtk_main_iteration.
I believe our main problem is transforms; for each path that a user
creates, Shoebot has to find its centerpoint to determine the transform.
The only way right now is using Cairo, since we have no other lib to
deal with this, AFAIK. This really sucks, using Cairo for such a simple
operation.
not only that, we have to do a lot of matrix parsing and re-generating
because there is no native Python lib for handling vector shapes,
matrices and data. Our implementation is a bit hackish -- nevertheless,
what we have works and looks great (and supports both Cairo's corner
transform mode as well as Nodebox's center mode), but the footprint of
all these operations begins to show when we use stacked transforms.
again, our solution would be a library to handle the Path, Matrix, and
all the primitive object types, which we could then use as a replacement
for most of the stuff in data.py. I've looked around a lot and couldn't
find anything similar in Python, but there's still hope -- lib2geom, the
C++ lib that powers Inkscape's path operations, already has some
bare-bones Python bindings.
i've talked to them on their Jabber channel and learned a bit how i
could help (njh is a great and patient guy :) -- it's mostly an effort
of reading the already existing bindings and writing the same for the
remaining ones. I've sent a patch already for binding a couple of
objects but there's still some work to be done.
the advantage of this would be, well, awesome: lib2geom supports boolean
operations between paths; for comparison, Nodebox's bezier library is
able to do that, albeit resulting in paths made out of only straight
lines, no curves. And well, if Inkscape uses it as a back-end, we can be
sure it will be a good bet.
i will be going on trying to add to their binding effort, but my almost
non-existent knowledge of C++ makes it tough on me; if there's anyone
willing to lend a hand to this, well, it would be really really great to
have a Python way to deal with beziers and related stuff, all this
besides how useful it would be for Shoebot.
if anyone's up for a sprint of sorts to work on this, it would be tops :-)
:r
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
Stuart Axon
2009-05-11 22:39:19 UTC
Permalink
I've been trying the Layer Profiler made by the pygame project -
and it's a lot easier (for me @ least) than the commandline profilers
http://www.pygame.org/project/1118/#

Just add
import profiler
profiler.pyglet() # set up GTK+ with pyglet
profiler.GProfiler().show() # show the window

Into the top of sbot and run it.
architetto francesco fantoni <hva - hermanitos verdes architetti>
2009-05-12 00:51:16 UTC
Permalink
refactored transforms.
I tried to clean and optimize the code, it seems a bit faster now, but
still slower than nodebox.
waiting for a complete rewrite (with lib2geom?)
I've found an affine transformations library using numpy and released
under BSD license, maybe i will try it.
francesco
Post by Stuart Axon
I wasn't sure if the slowdown was
- GTK/Pygtk
- The transforms (or other things we do with cairo that may be inefficient)
- Cairo in general
(or some combination of them).
To try and rule out the first one of these I tried drawing onto an opengl surface using pyglet
http://bitbucket.org/stuaxo/shoebot-pyglet/
It seems a little faster, but not significantly (this is subjective as I've only tested one bot so
far)
If you want to test your bot you can check it out, change size to 600,600 as it doesn't respect
size.
(only tested with sbot).
My next step is to try and convert some bots to pure cairo and see if they run faster, I'd
say it seems like your on the right track with making the transforms and matrices quicker
though.
If it turns out to be cairo then hopefully I can push some bugs in their direction.
----- Original Message ----
Sent: Monday, May 11, 2009 8:46:37 PM
Subject: Re: [shoebot-devel] why Shoebot can be slow, and a possible way out
Ricardo, I think you're absolutely right thinking that 2geom is perfect
for bezier handling, as you said it would add all sort of possibility
for boolean operations and mathematics on paths, and that would be
really great (and it seems rather powerful judging from inkscape).
What do you think about text and images? can be managed with lib2geom as
well, or we should have an alternative matrix handling for their
transformations?
I seriously doubt i could manage numpy well enough, i just scratched the
surface of it as i needed an array handling routine for the Image code
of shoebot and for video frames handling in sbopencv library, but I will
try to have a look at 2geom and numpy when I have the time, in order to
be able to possibly help when you decide to try the first implementation
of a new "next generation" shoebot ;)
ciao, francesco
Post by ricardo lafuente
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
I don't know much about lib2geom, i compiled it and had a look at the
examples, but i was scared by the total lack of documentation.
How about matrix handling using something like numpy? I don't know if it
could be done, nor if it could be faster than cairo or not...
True, it could definitely have some helper docs around. But since it's a
rather low-level lib, maybe that's not their priority. The examples are
pretty neat and run on Cairo -- try the python ones, they're practically
self-documenting. There's also something on 2geom on the Inkscape wiki
that briefly shows how it works (writing this off-line, try googling).
Something like Numpy would go better than Cairo, no doubt. I do remember
being pretty baffled by the complexity of the numpy concepts and quickly
got lost. But hey, you did work with it once, do you think it's doable?
(though i totally understand you won't want to place your eyes on an
affine matrix ever again :o)
On the other hand, 2geom have their own matrix implementation, and it's
already up and running on the Python bindings. Right now, they're just
missing bezier paths and other less simple primitives (like ellipses).
I'm personally very impressed by the whole of 2geom, especially after
playing with the existing python bindings on an ipython session. Not to
mention that Inkscape, which fundamentally relies on 2geom, regularly
contributes with patches.
Pardon my excitement, but i really think it's the missing piece of the
whole puzzle. Having a lib to take care of the abstract object data
(beziers, points, etc.) would fit everything together: Bot reads the
user scripts, 2geom objects are created, and finally passed to the
canvas, which will then call the appropriate commands on the back-end of
choice. And it would do away with our current mess regarding transforms
(a mess that works nonetheless, i'd emphasise :)
So yes, maybe it's not the kind of effort a hacker would fancy, but i
was hopeful i'd find someone who might be able to lend a hand.
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
Post by ricardo lafuente
hey all,
i was reading through data.py and was reminded of Stu's profiling
results with gtk_main_iteration.
I believe our main problem is transforms; for each path that a user
creates, Shoebot has to find its centerpoint to determine the transform.
The only way right now is using Cairo, since we have no other lib to
deal with this, AFAIK. This really sucks, using Cairo for such a simple
operation.
not only that, we have to do a lot of matrix parsing and re-generating
because there is no native Python lib for handling vector shapes,
matrices and data. Our implementation is a bit hackish -- nevertheless,
what we have works and looks great (and supports both Cairo's corner
transform mode as well as Nodebox's center mode), but the footprint of
all these operations begins to show when we use stacked transforms.
again, our solution would be a library to handle the Path, Matrix, and
all the primitive object types, which we could then use as a replacement
for most of the stuff in data.py. I've looked around a lot and couldn't
find anything similar in Python, but there's still hope -- lib2geom, the
C++ lib that powers Inkscape's path operations, already has some
bare-bones Python bindings.
i've talked to them on their Jabber channel and learned a bit how i
could help (njh is a great and patient guy :) -- it's mostly an effort
of reading the already existing bindings and writing the same for the
remaining ones. I've sent a patch already for binding a couple of
objects but there's still some work to be done.
the advantage of this would be, well, awesome: lib2geom supports boolean
operations between paths; for comparison, Nodebox's bezier library is
able to do that, albeit resulting in paths made out of only straight
lines, no curves. And well, if Inkscape uses it as a back-end, we can be
sure it will be a good bet.
i will be going on trying to add to their binding effort, but my almost
non-existent knowledge of C++ makes it tough on me; if there's anyone
willing to lend a hand to this, well, it would be really really great to
have a Python way to deal with beziers and related stuff, all this
besides how useful it would be for Shoebot.
if anyone's up for a sprint of sorts to work on this, it would be tops :-)
:r
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
francesco fantoni
2009-05-12 08:37:39 UTC
Permalink
bad news: I've a working implementation of transforms using pure numpy
array manipulation, and it seems slower than current cairo matrix
version...
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
refactored transforms.
I tried to clean and optimize the code, it seems a bit faster now, but
still slower than nodebox.
waiting for a complete rewrite (with lib2geom?)
I've found an affine transformations library using numpy and released
under BSD license, maybe i will try it.
francesco
--
architetto Francesco Fantoni
<HVA - Hermanitos Verdes Architetti>
l.go san giacomo, 38
I-41100 modena (italia)
tel.& fax. +39.059.217554
skype: hva_studio
e.mail: francesco-***@public.gmane.org
web: http://www.hv-a.com
Stuart Axon
2009-05-12 10:23:45 UTC
Permalink
Do you have a link to the repository, it'd still be useful to look at :)



----- Original Message ----
From: francesco fantoni <francesco-pB81QDORtWSUBmjqxMAjcGD2FQJk+8+***@public.gmane.org>
To: shoebot-devel-***@public.gmane.org
Sent: Tuesday, May 12, 2009 9:37:39 AM
Subject: Re: [shoebot-devel] why Shoebot can be slow, and a possible way out

bad news: I've a working implementation of transforms using pure numpy
array manipulation, and it seems slower than current cairo matrix
version...
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
refactored transforms.
I tried to clean and optimize the code, it seems a bit faster now, but
still slower than nodebox.
waiting for a complete rewrite (with lib2geom?)
I've found an affine transformations library using numpy and released
under BSD license, maybe i will try it.
francesco
--
architetto Francesco Fantoni
<HVA - Hermanitos Verdes Architetti>
l.go san giacomo, 38
I-41100 modena (italia)
tel.& fax. +39.059.217554
skype: hva_studio
e.mail: francesco-***@public.gmane.org
web: http://www.hv-a.com
ricardo lafuente
2009-05-12 11:42:00 UTC
Permalink
Post by francesco fantoni
bad news: I've a working implementation of transforms using pure numpy
array manipulation, and it seems slower than current cairo matrix
version...
wow, still, that's quite the effort!
i'm inclined to think that the bottleneck is on how we handle the
transforms -- there's a lot of stuff being calculated and re-calculated
(in some cases, needlessly, but was the only way to get it to work back
then). I've got to take a deeper look, and we could plan some
refactoring sprint soon.

on the bright side: after working some more on the 2geom bindings,
they're really close to usable. Some more testing, and maybe we can
start using 2geom in shoebot.
Post by francesco fantoni
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
refactored transforms.
I tried to clean and optimize the code, it seems a bit faster now, but
still slower than nodebox.
waiting for a complete rewrite (with lib2geom?)
I've found an affine transformations library using numpy and released
under BSD license, maybe i will try it.
francesco
------------------------------------------------------------------------
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
francesco fantoni
2009-05-12 12:50:57 UTC
Permalink
Post by ricardo lafuente
i'm inclined to think that the bottleneck is on how we handle the
transforms -- there's a lot of stuff being calculated and re-calculated
I thought the same. Each graphical object comes with its copy of the
transformations' chain up to the moment of its creation, and at
rendering time we're forced to go through a long chain of
transformations over and over for each object.
It would be faster to store a resulting matrix as state-attribute of the
grobs, calculating it only once and updating it whenever a
transformation command is parsed, but I'm afraid that is possibile only
for CORNER mode transformations I think, as the CENTER mode depends on
each object's coordinates.
I suppose you could calculate the resulting matrix for the CORNER mode
transformations and then "add" the CENTER mode ones on a per-object
basis, but the push-pop loops could really make it impossible to manage
and I suppose the order of application of each transformation could
matter as well, though I'm not sure...

Just for the records- the numpy variant is here:

http://freehg.org/u/hva/shoebot-affine/
Post by ricardo lafuente
on the bright side: after working some more on the 2geom bindings,
they're really close to usable. Some more testing, and maybe we can
start using 2geom in shoebot.
Post by francesco fantoni
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
refactored transforms.
I tried to clean and optimize the code, it seems a bit faster now, but
still slower than nodebox.
waiting for a complete rewrite (with lib2geom?)
I've found an affine transformations library using numpy and released
under BSD license, maybe i will try it.
francesco
------------------------------------------------------------------------
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
--
architetto Francesco Fantoni
<HVA - Hermanitos Verdes Architetti>
l.go san giacomo, 38
I-41100 modena (italia)
tel.& fax. +39.059.217554
skype: hva_studio
e.mail: francesco-***@public.gmane.org
web: http://www.hv-a.com
Stuart Axon
2009-05-12 13:25:23 UTC
Permalink
I think the CORNER, CENTER transformation stuff should still be manageable, we just need to push/pop objects which handle this, not individual matrices.


Do we still have to reverse the coordinates at the end ? - I wonder if we could use cairo.scale (or similar) to setup the coordinates we need (maybe we can use this to flip them).

Definitely storing the intermediate matrices preferably in some native format (lib2geom?) makes the a lot of sense too.


WRT refactoring -
I think it can definitely be made simpler.

External api:
A simple external api.

I'd imagine we'd be able to do:
from shoebot import bot, cairo_renderer

shoebot.run(botname, renderer= cairo_renderer)
cairo_frame = shoebot.next_frame()


- Probably we should move a lot of the colour transformation methods out of data and into something like colorsystems.py

Strings vs references:
I think there are a few places we could use references instead of
if 'somestring'
pass
elif 'someotherstring'
pass

Namespace stuff:
copyattrs seems to show up quite high in the profiling, I wonder if we could do this in a different way avoiding isinstance
(I wonder if we could also avoid setattr and getattr too)


This sort of thing can be avoided:
if namespace['draw']: ## from memory
We can check if it exists once in the bot at the beginning and add an empty stub if it doesn't.


Window resizing stuff:
We should be able to resize the window (probably everything inside could just scale like nodebox(?))

We need to sort out size()
I don't know what nodebox does if you call it in the middle of a script, but I guess we would just reset the canvas to the new size... being able to do this might avoid some of the weirdness around here.


-- Sorry if this is a bit of a braindump, it's still in my head from playing with the code last night :)
... + hopefully I will get round to looking at some of these myself.


----- Original Message ----
From: francesco fantoni <francesco-pB81QDORtWSUBmjqxMAjcGD2FQJk+8+***@public.gmane.org>
To: shoebot-devel-***@public.gmane.org
Sent: Tuesday, May 12, 2009 1:50:57 PM
Subject: Re: [shoebot-devel] why Shoebot can be slow, and a possible way out
Post by ricardo lafuente
i'm inclined to think that the bottleneck is on how we handle the
transforms -- there's a lot of stuff being calculated and re-calculated
I thought the same. Each graphical object comes with its copy of the
transformations' chain up to the moment of its creation, and at
rendering time we're forced to go through a long chain of
transformations over and over for each object.
It would be faster to store a resulting matrix as state-attribute of the
grobs, calculating it only once and updating it whenever a
transformation command is parsed, but I'm afraid that is possibile only
for CORNER mode transformations I think, as the CENTER mode depends on
each object's coordinates.
I suppose you could calculate the resulting matrix for the CORNER mode
transformations and then "add" the CENTER mode ones on a per-object
basis, but the push-pop loops could really make it impossible to manage
and I suppose the order of application of each transformation could
matter as well, though I'm not sure...

Just for the records- the numpy variant is here:

http://freehg.org/u/hva/shoebot-affine/
Post by ricardo lafuente
on the bright side: after working some more on the 2geom bindings,
they're really close to usable. Some more testing, and maybe we can
start using 2geom in shoebot.
Post by francesco fantoni
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
refactored transforms.
I tried to clean and optimize the code, it seems a bit faster now, but
still slower than nodebox.
waiting for a complete rewrite (with lib2geom?)
I've found an affine transformations library using numpy and released
under BSD license, maybe i will try it.
francesco
------------------------------------------------------------------------
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
--
architetto Francesco Fantoni
<HVA - Hermanitos Verdes Architetti>
l.go san giacomo, 38
I-41100 modena (italia)
tel.& fax. +39.059.217554
skype: hva_studio
e.mail: francesco-***@public.gmane.org
web: http://www.hv-a.com
ricardo lafuente
2009-05-13 14:34:35 UTC
Permalink
wow Stuart, thanks for all the great points; i'll just comment a bit and
later i'll add them to our tracker for the path to 0.3.
Post by Stuart Axon
I think the CORNER, CENTER transformation stuff should still be manageable, we just need to push/pop objects which handle this, not individual matrices.
true; the difference is just translating and applying the transform
(CENTER), as opposed to applying the transform only (CORNER).
most of this was made harder because of our hacky solution to get the
path bounds (= dimensions and center). I think that's probably one of
the bottlenecks, now that i think of it: to determine the path bounds,
we're creating a new cairo context, drawing the object, measuring it and
then deleting everything. This is total overkill, and something that
should be made a lot more straightforward when we try doing it via 2geom
(more on this below).
Post by Stuart Axon
Do we still have to reverse the coordinates at the end ? - I wonder if we could use cairo.scale (or similar) to setup the coordinates we need (maybe we can use this to flip them).
do we do this coordinate reversal? I can't recall where, to be honest...

...but maybe OT, this is an issue to be dealt with when DrawBot is to be
implemented, since its coordinate system has the origin at the
bottom-left corner. Not sure how easy it will be.
Post by Stuart Axon
Definitely storing the intermediate matrices preferably in some native format (lib2geom?) makes the a lot of sense too.
yes, lib2geom has all we need for matrices. Does everything cairo.Matrix
does from what i saw.
Post by Stuart Axon
WRT refactoring -
I think it can definitely be made simpler.
A simple external api.
from shoebot import bot, cairo_renderer
shoebot.run(botname, renderer= cairo_renderer)
cairo_frame = shoebot.next_frame()
great point, hadn't thought of this.
something this reminds me is how Processing handles the backend choice:
you specify it in size(), as in

size(1024, 768, OPENGL)

once we have more backends, this might be a way to go too. But your
solution is great when not reading an external script and calling bot
commands from python.
Post by Stuart Axon
- Probably we should move a lot of the colour transformation methods out of data and into something like colorsystems.py
totally, as a matter of fact we should look into our file structure:
- gtkui contains both the windowed mode code (when you specify -w) as
well as the IDE. These should definitely be separated.
- the bot and canvas stuff should get out of the __init__.py file and be
moved to something like a shoebot.core module.
- libs should be importable through a module, e.g. 'from shoebot.libs
import sbopencv' or something. Not sure if this is allowed already.
- maybe breaking apart data.py would be in order -- separating
primitives, paths, transforms and colors.
Post by Stuart Axon
I think there are a few places we could use references instead of
if 'somestring'
pass
elif 'someotherstring'
pass
yes! We ought to make a list of where this is happening.
Post by Stuart Axon
copyattrs seems to show up quite high in the profiling, I wonder if we could do this in a different way avoiding isinstance
(I wonder if we could also avoid setattr and getattr too)
good to know; it was something copied directly from Nodebox. Wondering
if there's faster methods to do it. Anyway, it's a function that's
called a *lot* of times, so it's bound to be abundant when looking at
debug output.
Post by Stuart Axon
if namespace['draw']: ## from memory
We can check if it exists once in the bot at the beginning and add an empty stub if it doesn't.
good point
Post by Stuart Axon
We should be able to resize the window (probably everything inside could just scale like nodebox(?))
yes, definitely; this would imply having a 'view' instead of the window
replicating the Cairo context, so that scaling wouldn't change the final
output. That would also allow for zooming, panning, etc.
Post by Stuart Axon
We need to sort out size()
I don't know what nodebox does if you call it in the middle of a script, but I guess we would just reset the canvas to the new size... being able to do this might avoid some of the weirdness around here.
yeah -- it does have that issue with calling size() using variables
instead of absolute values, but that's something we can warn against.
but you're right, at the very least it should issue a warning and fail
silently when called more than once.

:r
Post by Stuart Axon
-- Sorry if this is a bit of a braindump, it's still in my head from playing with the code last night :)
... + hopefully I will get round to looking at some of these myself.
----- Original Message ----
Sent: Tuesday, May 12, 2009 1:50:57 PM
Subject: Re: [shoebot-devel] why Shoebot can be slow, and a possible way out
Post by ricardo lafuente
i'm inclined to think that the bottleneck is on how we handle the
transforms -- there's a lot of stuff being calculated and re-calculated
I thought the same. Each graphical object comes with its copy of the
transformations' chain up to the moment of its creation, and at
rendering time we're forced to go through a long chain of
transformations over and over for each object.
It would be faster to store a resulting matrix as state-attribute of the
grobs, calculating it only once and updating it whenever a
transformation command is parsed, but I'm afraid that is possibile only
for CORNER mode transformations I think, as the CENTER mode depends on
each object's coordinates.
I suppose you could calculate the resulting matrix for the CORNER mode
transformations and then "add" the CENTER mode ones on a per-object
basis, but the push-pop loops could really make it impossible to manage
and I suppose the order of application of each transformation could
matter as well, though I'm not sure...
http://freehg.org/u/hva/shoebot-affine/
Post by ricardo lafuente
on the bright side: after working some more on the 2geom bindings,
they're really close to usable. Some more testing, and maybe we can
start using 2geom in shoebot.
Post by francesco fantoni
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
refactored transforms.
I tried to clean and optimize the code, it seems a bit faster now, but
still slower than nodebox.
waiting for a complete rewrite (with lib2geom?)
I've found an affine transformations library using numpy and released
under BSD license, maybe i will try it.
francesco
------------------------------------------------------------------------
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
architetto francesco fantoni <hva - hermanitos verdes architetti>
2009-05-13 18:33:52 UTC
Permalink
how does lib2geom interacts with text (pango) and images? could them be
managed as well? text can obviously be converted to paths (though it
would be nice if lib2geom itself tokk care of this), but how about
images?
Post by ricardo lafuente
wow Stuart, thanks for all the great points; i'll just comment a bit and
later i'll add them to our tracker for the path to 0.3.
Post by Stuart Axon
I think the CORNER, CENTER transformation stuff should still be manageable, we just need to push/pop objects which handle this, not individual matrices.
true; the difference is just translating and applying the transform
(CENTER), as opposed to applying the transform only (CORNER).
most of this was made harder because of our hacky solution to get the
path bounds (= dimensions and center). I think that's probably one of
the bottlenecks, now that i think of it: to determine the path bounds,
we're creating a new cairo context, drawing the object, measuring it and
then deleting everything. This is total overkill, and something that
should be made a lot more straightforward when we try doing it via 2geom
(more on this below).
Post by Stuart Axon
Do we still have to reverse the coordinates at the end ? - I wonder if we could use cairo.scale (or similar) to setup the coordinates we need (maybe we can use this to flip them).
do we do this coordinate reversal? I can't recall where, to be honest...
...but maybe OT, this is an issue to be dealt with when DrawBot is to be
implemented, since its coordinate system has the origin at the
bottom-left corner. Not sure how easy it will be.
Post by Stuart Axon
Definitely storing the intermediate matrices preferably in some native format (lib2geom?) makes the a lot of sense too.
yes, lib2geom has all we need for matrices. Does everything cairo.Matrix
does from what i saw.
Post by Stuart Axon
WRT refactoring -
I think it can definitely be made simpler.
A simple external api.
from shoebot import bot, cairo_renderer
shoebot.run(botname, renderer= cairo_renderer)
cairo_frame = shoebot.next_frame()
great point, hadn't thought of this.
you specify it in size(), as in
size(1024, 768, OPENGL)
once we have more backends, this might be a way to go too. But your
solution is great when not reading an external script and calling bot
commands from python.
Post by Stuart Axon
- Probably we should move a lot of the colour transformation methods out of data and into something like colorsystems.py
- gtkui contains both the windowed mode code (when you specify -w) as
well as the IDE. These should definitely be separated.
- the bot and canvas stuff should get out of the __init__.py file and be
moved to something like a shoebot.core module.
- libs should be importable through a module, e.g. 'from shoebot.libs
import sbopencv' or something. Not sure if this is allowed already.
- maybe breaking apart data.py would be in order -- separating
primitives, paths, transforms and colors.
Post by Stuart Axon
I think there are a few places we could use references instead of
if 'somestring'
pass
elif 'someotherstring'
pass
yes! We ought to make a list of where this is happening.
Post by Stuart Axon
copyattrs seems to show up quite high in the profiling, I wonder if we could do this in a different way avoiding isinstance
(I wonder if we could also avoid setattr and getattr too)
good to know; it was something copied directly from Nodebox. Wondering
if there's faster methods to do it. Anyway, it's a function that's
called a *lot* of times, so it's bound to be abundant when looking at
debug output.
Post by Stuart Axon
if namespace['draw']: ## from memory
We can check if it exists once in the bot at the beginning and add an empty stub if it doesn't.
good point
Post by Stuart Axon
We should be able to resize the window (probably everything inside could just scale like nodebox(?))
yes, definitely; this would imply having a 'view' instead of the window
replicating the Cairo context, so that scaling wouldn't change the final
output. That would also allow for zooming, panning, etc.
Post by Stuart Axon
We need to sort out size()
I don't know what nodebox does if you call it in the middle of a script, but I guess we would just reset the canvas to the new size... being able to do this might avoid some of the weirdness around here.
yeah -- it does have that issue with calling size() using variables
instead of absolute values, but that's something we can warn against.
but you're right, at the very least it should issue a warning and fail
silently when called more than once.
:r
Post by Stuart Axon
-- Sorry if this is a bit of a braindump, it's still in my head from playing with the code last night :)
... + hopefully I will get round to looking at some of these myself.
----- Original Message ----
Sent: Tuesday, May 12, 2009 1:50:57 PM
Subject: Re: [shoebot-devel] why Shoebot can be slow, and a possible way out
Post by ricardo lafuente
i'm inclined to think that the bottleneck is on how we handle the
transforms -- there's a lot of stuff being calculated and re-calculated
I thought the same. Each graphical object comes with its copy of the
transformations' chain up to the moment of its creation, and at
rendering time we're forced to go through a long chain of
transformations over and over for each object.
It would be faster to store a resulting matrix as state-attribute of the
grobs, calculating it only once and updating it whenever a
transformation command is parsed, but I'm afraid that is possibile only
for CORNER mode transformations I think, as the CENTER mode depends on
each object's coordinates.
I suppose you could calculate the resulting matrix for the CORNER mode
transformations and then "add" the CENTER mode ones on a per-object
basis, but the push-pop loops could really make it impossible to manage
and I suppose the order of application of each transformation could
matter as well, though I'm not sure...
http://freehg.org/u/hva/shoebot-affine/
Post by ricardo lafuente
on the bright side: after working some more on the 2geom bindings,
they're really close to usable. Some more testing, and maybe we can
start using 2geom in shoebot.
Post by francesco fantoni
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
refactored transforms.
I tried to clean and optimize the code, it seems a bit faster now, but
still slower than nodebox.
waiting for a complete rewrite (with lib2geom?)
I've found an affine transformations library using numpy and released
under BSD license, maybe i will try it.
francesco
------------------------------------------------------------------------
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
ricardo lafuente
2009-05-17 17:06:05 UTC
Permalink
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
how does lib2geom interacts with text (pango) and images? could them be
managed as well? text can obviously be converted to paths (though it
would be nice if lib2geom itself tokk care of this), but how about
images?
well, it doesn't!

what it does is provide classes for shapes (paths, primitives) and some
other useful elements when working with vectors (matrices, functions for
calculating intersections, etc etc.)

so, our use for it would be to change our BezierPath class (and maybe
the primitives) to work with the 2geom classes. My idea would be to
subclass the 2geom Path, and from there add some more methods (e.g.
BezierPath.contains(point)).

WRT text classes, i think we could leave them as they are; maybe
determining its bounding rect using 2geom would be useful for
calculating the text bounds. On the other hand, of course we can use
2geom for outlined text paths.

For images, maybe we could have a 2geom rect corresponding to its
dimensions.

I hope i'm making sense here -- essentially, text and images remain a
job for the renderer, not 2geom; however, it's probably useful if we
save their bounding boxes as 2geom rects -- this would free us from
having to use Cairo to determine path/object bounds, as we do now (and
which is probably one of our bottlenecks.)

Regarding transforms, 2geom will help too, however i kinda wonder if
they're using Cairo behind the scenes...
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
Post by ricardo lafuente
wow Stuart, thanks for all the great points; i'll just comment a bit and
later i'll add them to our tracker for the path to 0.3.
Post by Stuart Axon
I think the CORNER, CENTER transformation stuff should still be manageable, we just need to push/pop objects which handle this, not individual matrices.
true; the difference is just translating and applying the transform
(CENTER), as opposed to applying the transform only (CORNER).
most of this was made harder because of our hacky solution to get the
path bounds (= dimensions and center). I think that's probably one of
the bottlenecks, now that i think of it: to determine the path bounds,
we're creating a new cairo context, drawing the object, measuring it and
then deleting everything. This is total overkill, and something that
should be made a lot more straightforward when we try doing it via 2geom
(more on this below).
Post by Stuart Axon
Do we still have to reverse the coordinates at the end ? - I wonder if we could use cairo.scale (or similar) to setup the coordinates we need (maybe we can use this to flip them).
do we do this coordinate reversal? I can't recall where, to be honest...
...but maybe OT, this is an issue to be dealt with when DrawBot is to be
implemented, since its coordinate system has the origin at the
bottom-left corner. Not sure how easy it will be.
Post by Stuart Axon
Definitely storing the intermediate matrices preferably in some native format (lib2geom?) makes the a lot of sense too.
yes, lib2geom has all we need for matrices. Does everything cairo.Matrix
does from what i saw.
Post by Stuart Axon
WRT refactoring -
I think it can definitely be made simpler.
A simple external api.
from shoebot import bot, cairo_renderer
shoebot.run(botname, renderer= cairo_renderer)
cairo_frame = shoebot.next_frame()
great point, hadn't thought of this.
you specify it in size(), as in
size(1024, 768, OPENGL)
once we have more backends, this might be a way to go too. But your
solution is great when not reading an external script and calling bot
commands from python.
Post by Stuart Axon
- Probably we should move a lot of the colour transformation methods out of data and into something like colorsystems.py
- gtkui contains both the windowed mode code (when you specify -w) as
well as the IDE. These should definitely be separated.
- the bot and canvas stuff should get out of the __init__.py file and be
moved to something like a shoebot.core module.
- libs should be importable through a module, e.g. 'from shoebot.libs
import sbopencv' or something. Not sure if this is allowed already.
- maybe breaking apart data.py would be in order -- separating
primitives, paths, transforms and colors.
Post by Stuart Axon
I think there are a few places we could use references instead of
if 'somestring'
pass
elif 'someotherstring'
pass
yes! We ought to make a list of where this is happening.
Post by Stuart Axon
copyattrs seems to show up quite high in the profiling, I wonder if we could do this in a different way avoiding isinstance
(I wonder if we could also avoid setattr and getattr too)
good to know; it was something copied directly from Nodebox. Wondering
if there's faster methods to do it. Anyway, it's a function that's
called a *lot* of times, so it's bound to be abundant when looking at
debug output.
Post by Stuart Axon
if namespace['draw']: ## from memory
We can check if it exists once in the bot at the beginning and add an empty stub if it doesn't.
good point
Post by Stuart Axon
We should be able to resize the window (probably everything inside could just scale like nodebox(?))
yes, definitely; this would imply having a 'view' instead of the window
replicating the Cairo context, so that scaling wouldn't change the final
output. That would also allow for zooming, panning, etc.
Post by Stuart Axon
We need to sort out size()
I don't know what nodebox does if you call it in the middle of a script, but I guess we would just reset the canvas to the new size... being able to do this might avoid some of the weirdness around here.
yeah -- it does have that issue with calling size() using variables
instead of absolute values, but that's something we can warn against.
but you're right, at the very least it should issue a warning and fail
silently when called more than once.
:r
Post by Stuart Axon
-- Sorry if this is a bit of a braindump, it's still in my head from playing with the code last night :)
... + hopefully I will get round to looking at some of these myself.
----- Original Message ----
Sent: Tuesday, May 12, 2009 1:50:57 PM
Subject: Re: [shoebot-devel] why Shoebot can be slow, and a possible way out
Post by ricardo lafuente
i'm inclined to think that the bottleneck is on how we handle the
transforms -- there's a lot of stuff being calculated and re-calculated
I thought the same. Each graphical object comes with its copy of the
transformations' chain up to the moment of its creation, and at
rendering time we're forced to go through a long chain of
transformations over and over for each object.
It would be faster to store a resulting matrix as state-attribute of the
grobs, calculating it only once and updating it whenever a
transformation command is parsed, but I'm afraid that is possibile only
for CORNER mode transformations I think, as the CENTER mode depends on
each object's coordinates.
I suppose you could calculate the resulting matrix for the CORNER mode
transformations and then "add" the CENTER mode ones on a per-object
basis, but the push-pop loops could really make it impossible to manage
and I suppose the order of application of each transformation could
matter as well, though I'm not sure...
http://freehg.org/u/hva/shoebot-affine/
Post by ricardo lafuente
on the bright side: after working some more on the 2geom bindings,
they're really close to usable. Some more testing, and maybe we can
start using 2geom in shoebot.
Post by francesco fantoni
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
refactored transforms.
I tried to clean and optimize the code, it seems a bit faster now, but
still slower than nodebox.
waiting for a complete rewrite (with lib2geom?)
I've found an affine transformations library using numpy and released
under BSD license, maybe i will try it.
francesco
------------------------------------------------------------------------
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
francesco fantoni
2009-05-19 12:48:09 UTC
Permalink
seems it vould work like that. I only wish there was some documentation
for 2geom, as I've been looking at the provided examples, but it felt a
bit obscure to me.

ùIl giorno dom, 17/05/2009 alle 18.06 +0100, ricardo lafuente ha
Post by ricardo lafuente
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
how does lib2geom interacts with text (pango) and images? could them be
managed as well? text can obviously be converted to paths (though it
would be nice if lib2geom itself tokk care of this), but how about
images?
well, it doesn't!
what it does is provide classes for shapes (paths, primitives) and some
other useful elements when working with vectors (matrices, functions for
calculating intersections, etc etc.)
so, our use for it would be to change our BezierPath class (and maybe
the primitives) to work with the 2geom classes. My idea would be to
subclass the 2geom Path, and from there add some more methods (e.g.
BezierPath.contains(point)).
WRT text classes, i think we could leave them as they are; maybe
determining its bounding rect using 2geom would be useful for
calculating the text bounds. On the other hand, of course we can use
2geom for outlined text paths.
For images, maybe we could have a 2geom rect corresponding to its
dimensions.
I hope i'm making sense here -- essentially, text and images remain a
job for the renderer, not 2geom; however, it's probably useful if we
save their bounding boxes as 2geom rects -- this would free us from
having to use Cairo to determine path/object bounds, as we do now (and
which is probably one of our bottlenecks.)
Regarding transforms, 2geom will help too, however i kinda wonder if
they're using Cairo behind the scenes...
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
Post by ricardo lafuente
wow Stuart, thanks for all the great points; i'll just comment a bit and
later i'll add them to our tracker for the path to 0.3.
Post by Stuart Axon
I think the CORNER, CENTER transformation stuff should still be manageable, we just need to push/pop objects which handle this, not individual matrices.
true; the difference is just translating and applying the transform
(CENTER), as opposed to applying the transform only (CORNER).
most of this was made harder because of our hacky solution to get the
path bounds (= dimensions and center). I think that's probably one of
the bottlenecks, now that i think of it: to determine the path bounds,
we're creating a new cairo context, drawing the object, measuring it and
then deleting everything. This is total overkill, and something that
should be made a lot more straightforward when we try doing it via 2geom
(more on this below).
Post by Stuart Axon
Do we still have to reverse the coordinates at the end ? - I wonder if we could use cairo.scale (or similar) to setup the coordinates we need (maybe we can use this to flip them).
do we do this coordinate reversal? I can't recall where, to be honest...
...but maybe OT, this is an issue to be dealt with when DrawBot is to be
implemented, since its coordinate system has the origin at the
bottom-left corner. Not sure how easy it will be.
Post by Stuart Axon
Definitely storing the intermediate matrices preferably in some native format (lib2geom?) makes the a lot of sense too.
yes, lib2geom has all we need for matrices. Does everything cairo.Matrix
does from what i saw.
Post by Stuart Axon
WRT refactoring -
I think it can definitely be made simpler.
A simple external api.
from shoebot import bot, cairo_renderer
shoebot.run(botname, renderer= cairo_renderer)
cairo_frame = shoebot.next_frame()
great point, hadn't thought of this.
you specify it in size(), as in
size(1024, 768, OPENGL)
once we have more backends, this might be a way to go too. But your
solution is great when not reading an external script and calling bot
commands from python.
Post by Stuart Axon
- Probably we should move a lot of the colour transformation methods out of data and into something like colorsystems.py
- gtkui contains both the windowed mode code (when you specify -w) as
well as the IDE. These should definitely be separated.
- the bot and canvas stuff should get out of the __init__.py file and be
moved to something like a shoebot.core module.
- libs should be importable through a module, e.g. 'from shoebot.libs
import sbopencv' or something. Not sure if this is allowed already.
- maybe breaking apart data.py would be in order -- separating
primitives, paths, transforms and colors.
Post by Stuart Axon
I think there are a few places we could use references instead of
if 'somestring'
pass
elif 'someotherstring'
pass
yes! We ought to make a list of where this is happening.
Post by Stuart Axon
copyattrs seems to show up quite high in the profiling, I wonder if we could do this in a different way avoiding isinstance
(I wonder if we could also avoid setattr and getattr too)
good to know; it was something copied directly from Nodebox. Wondering
if there's faster methods to do it. Anyway, it's a function that's
called a *lot* of times, so it's bound to be abundant when looking at
debug output.
Post by Stuart Axon
if namespace['draw']: ## from memory
We can check if it exists once in the bot at the beginning and add an empty stub if it doesn't.
good point
Post by Stuart Axon
We should be able to resize the window (probably everything inside could just scale like nodebox(?))
yes, definitely; this would imply having a 'view' instead of the window
replicating the Cairo context, so that scaling wouldn't change the final
output. That would also allow for zooming, panning, etc.
Post by Stuart Axon
We need to sort out size()
I don't know what nodebox does if you call it in the middle of a script, but I guess we would just reset the canvas to the new size... being able to do this might avoid some of the weirdness around here.
yeah -- it does have that issue with calling size() using variables
instead of absolute values, but that's something we can warn against.
but you're right, at the very least it should issue a warning and fail
silently when called more than once.
:r
Post by Stuart Axon
-- Sorry if this is a bit of a braindump, it's still in my head from playing with the code last night :)
... + hopefully I will get round to looking at some of these myself.
----- Original Message ----
Sent: Tuesday, May 12, 2009 1:50:57 PM
Subject: Re: [shoebot-devel] why Shoebot can be slow, and a possible way out
Post by ricardo lafuente
i'm inclined to think that the bottleneck is on how we handle the
transforms -- there's a lot of stuff being calculated and re-calculated
I thought the same. Each graphical object comes with its copy of the
transformations' chain up to the moment of its creation, and at
rendering time we're forced to go through a long chain of
transformations over and over for each object.
It would be faster to store a resulting matrix as state-attribute of the
grobs, calculating it only once and updating it whenever a
transformation command is parsed, but I'm afraid that is possibile only
for CORNER mode transformations I think, as the CENTER mode depends on
each object's coordinates.
I suppose you could calculate the resulting matrix for the CORNER mode
transformations and then "add" the CENTER mode ones on a per-object
basis, but the push-pop loops could really make it impossible to manage
and I suppose the order of application of each transformation could
matter as well, though I'm not sure...
http://freehg.org/u/hva/shoebot-affine/
Post by ricardo lafuente
on the bright side: after working some more on the 2geom bindings,
they're really close to usable. Some more testing, and maybe we can
start using 2geom in shoebot.
Post by francesco fantoni
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
refactored transforms.
I tried to clean and optimize the code, it seems a bit faster now, but
still slower than nodebox.
waiting for a complete rewrite (with lib2geom?)
I've found an affine transformations library using numpy and released
under BSD license, maybe i will try it.
francesco
------------------------------------------------------------------------
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
--
architetto Francesco Fantoni
<HVA - Hermanitos Verdes Architetti>
l.go san giacomo, 38
I-41100 modena (italia)
tel.& fax. +39.059.217554
skype: hva_studio
e.mail: ***@hv-a.com
web: http://www.hv-a.com
ricardo lafuente
2009-05-27 04:49:28 UTC
Permalink
there are doxygen-generated docs (run doxygen on the root of the
lib2geom source tree); lacking in some aspects, but useful anyway.

but no doubt a simple 2geom 'HOWTO Get Started' would help a *lot*. No
surprise there isn't any, given that you'd need to be a C++ hacker to
dive in into the library. Njh (one of the 2geom gods) remarked that
Shoebot would be a most useful tool for a tutorial-like introduction, as
well as visually showing the workings of 2geom. So one can be of help
here without touching the nitty-gritty details its codebase.

i'd say the best way to dive in, if you're up to getting your hands
dirty with the py bindings, is to check out the existing code:
* python bindings are in src/2geom/py2geom (which are written based on
the .h header files on src/2geom)
* some toys (including some in python) are in the toys/ folder, useful
to see how it works

but most of the stuff (path, matrix, point, circle, ellipse) is in good
shape already, so i'd say the useful effort would be to actually use
2geom in some example scripts.

my advice would be checking out their jabber channel (lib2geom at
conference.gristle.org). They're really, really helpful over there, and
willing to guide anyone to get what's needed.

i'll be on the 2geom porting effort as soon as i'm settled over at the
tail of India (which is also the reason for my late reply)
Post by francesco fantoni
seems it vould work like that. I only wish there was some documentation
for 2geom, as I've been looking at the provided examples, but it felt a
bit obscure to me.
ùIl giorno dom, 17/05/2009 alle 18.06 +0100, ricardo lafuente ha
Post by ricardo lafuente
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
how does lib2geom interacts with text (pango) and images? could them be
managed as well? text can obviously be converted to paths (though it
would be nice if lib2geom itself tokk care of this), but how about
images?
well, it doesn't!
what it does is provide classes for shapes (paths, primitives) and some
other useful elements when working with vectors (matrices, functions for
calculating intersections, etc etc.)
so, our use for it would be to change our BezierPath class (and maybe
the primitives) to work with the 2geom classes. My idea would be to
subclass the 2geom Path, and from there add some more methods (e.g.
BezierPath.contains(point)).
WRT text classes, i think we could leave them as they are; maybe
determining its bounding rect using 2geom would be useful for
calculating the text bounds. On the other hand, of course we can use
2geom for outlined text paths.
For images, maybe we could have a 2geom rect corresponding to its
dimensions.
I hope i'm making sense here -- essentially, text and images remain a
job for the renderer, not 2geom; however, it's probably useful if we
save their bounding boxes as 2geom rects -- this would free us from
having to use Cairo to determine path/object bounds, as we do now (and
which is probably one of our bottlenecks.)
Regarding transforms, 2geom will help too, however i kinda wonder if
they're using Cairo behind the scenes...
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
Post by ricardo lafuente
wow Stuart, thanks for all the great points; i'll just comment a bit and
later i'll add them to our tracker for the path to 0.3.
Post by Stuart Axon
I think the CORNER, CENTER transformation stuff should still be manageable, we just need to push/pop objects which handle this, not individual matrices.
true; the difference is just translating and applying the transform
(CENTER), as opposed to applying the transform only (CORNER).
most of this was made harder because of our hacky solution to get the
path bounds (= dimensions and center). I think that's probably one of
the bottlenecks, now that i think of it: to determine the path bounds,
we're creating a new cairo context, drawing the object, measuring it and
then deleting everything. This is total overkill, and something that
should be made a lot more straightforward when we try doing it via 2geom
(more on this below).
Post by Stuart Axon
Do we still have to reverse the coordinates at the end ? - I wonder if we could use cairo.scale (or similar) to setup the coordinates we need (maybe we can use this to flip them).
do we do this coordinate reversal? I can't recall where, to be honest...
...but maybe OT, this is an issue to be dealt with when DrawBot is to be
implemented, since its coordinate system has the origin at the
bottom-left corner. Not sure how easy it will be.
Post by Stuart Axon
Definitely storing the intermediate matrices preferably in some native format (lib2geom?) makes the a lot of sense too.
yes, lib2geom has all we need for matrices. Does everything cairo.Matrix
does from what i saw.
Post by Stuart Axon
WRT refactoring -
I think it can definitely be made simpler.
A simple external api.
from shoebot import bot, cairo_renderer
shoebot.run(botname, renderer= cairo_renderer)
cairo_frame = shoebot.next_frame()
great point, hadn't thought of this.
you specify it in size(), as in
size(1024, 768, OPENGL)
once we have more backends, this might be a way to go too. But your
solution is great when not reading an external script and calling bot
commands from python.
Post by Stuart Axon
- Probably we should move a lot of the colour transformation methods out of data and into something like colorsystems.py
- gtkui contains both the windowed mode code (when you specify -w) as
well as the IDE. These should definitely be separated.
- the bot and canvas stuff should get out of the __init__.py file and be
moved to something like a shoebot.core module.
- libs should be importable through a module, e.g. 'from shoebot.libs
import sbopencv' or something. Not sure if this is allowed already.
- maybe breaking apart data.py would be in order -- separating
primitives, paths, transforms and colors.
Post by Stuart Axon
I think there are a few places we could use references instead of
if 'somestring'
pass
elif 'someotherstring'
pass
yes! We ought to make a list of where this is happening.
Post by Stuart Axon
copyattrs seems to show up quite high in the profiling, I wonder if we could do this in a different way avoiding isinstance
(I wonder if we could also avoid setattr and getattr too)
good to know; it was something copied directly from Nodebox. Wondering
if there's faster methods to do it. Anyway, it's a function that's
called a *lot* of times, so it's bound to be abundant when looking at
debug output.
Post by Stuart Axon
if namespace['draw']: ## from memory
We can check if it exists once in the bot at the beginning and add an empty stub if it doesn't.
good point
Post by Stuart Axon
We should be able to resize the window (probably everything inside could just scale like nodebox(?))
yes, definitely; this would imply having a 'view' instead of the window
replicating the Cairo context, so that scaling wouldn't change the final
output. That would also allow for zooming, panning, etc.
Post by Stuart Axon
We need to sort out size()
I don't know what nodebox does if you call it in the middle of a script, but I guess we would just reset the canvas to the new size... being able to do this might avoid some of the weirdness around here.
yeah -- it does have that issue with calling size() using variables
instead of absolute values, but that's something we can warn against.
but you're right, at the very least it should issue a warning and fail
silently when called more than once.
:r
Post by Stuart Axon
-- Sorry if this is a bit of a braindump, it's still in my head from playing with the code last night :)
... + hopefully I will get round to looking at some of these myself.
----- Original Message ----
Sent: Tuesday, May 12, 2009 1:50:57 PM
Subject: Re: [shoebot-devel] why Shoebot can be slow, and a possible way out
Post by ricardo lafuente
i'm inclined to think that the bottleneck is on how we handle the
transforms -- there's a lot of stuff being calculated and re-calculated
I thought the same. Each graphical object comes with its copy of the
transformations' chain up to the moment of its creation, and at
rendering time we're forced to go through a long chain of
transformations over and over for each object.
It would be faster to store a resulting matrix as state-attribute of the
grobs, calculating it only once and updating it whenever a
transformation command is parsed, but I'm afraid that is possibile only
for CORNER mode transformations I think, as the CENTER mode depends on
each object's coordinates.
I suppose you could calculate the resulting matrix for the CORNER mode
transformations and then "add" the CENTER mode ones on a per-object
basis, but the push-pop loops could really make it impossible to manage
and I suppose the order of application of each transformation could
matter as well, though I'm not sure...
http://freehg.org/u/hva/shoebot-affine/
Post by ricardo lafuente
on the bright side: after working some more on the 2geom bindings,
they're really close to usable. Some more testing, and maybe we can
start using 2geom in shoebot.
Post by francesco fantoni
Post by architetto francesco fantoni <hva - hermanitos verdes architetti>
refactored transforms.
I tried to clean and optimize the code, it seems a bit faster now, but
still slower than nodebox.
waiting for a complete rewrite (with lib2geom?)
I've found an affine transformations library using numpy and released
under BSD license, maybe i will try it.
francesco
------------------------------------------------------------------------
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net
Loading...