Toybot branch up

[Stuart Axon]

Here: http://bitbucket.org/stuaxo/toybot/


At the moment theres just shoebot.py.

If you run it, you'll get 10 SVGs.
The Bot itself is embedded at the end of the code for now.

All thats working right now is:

background(r, g, b)
size()

rect()
FRAME
fill(r, g, b)



Heres the TODO:


1
Implement transforms.

1a
- The transform should be tracked as the bot runs, with just the absolute
coordinates being output to the draw queue at each stage.
Reference transform implementation:
http://dev.nodebox.net/browser/nodebox-java/branches/rewrite/src/java/net/nodebox/graphics/Grob.java
getCenteredTransform()

1b.
Write test bots, to check transforms on nodebox work in the same way as here.



2
Get output working to a window.

2a
Implement Either GTK, or use something else for output, e.g. pyglet

2b
Test performance against shoebot classic.


3
Review external API.

Aims:
- It should be as easy as possible to pass in a Cairo context and get a frame
drawn on it.

- Possibly be able to iterate through frames on a bot.


4
Integrate new code.

4a
Start new shoebot branch

4b
Move code that is working into branch

4c
Stub out other methods with NotImplemented


S++

[ricardo lafuente]

Some notes after a quick skim:

- I love the callable passing idea
- Great that you used faux-private variables (_variable), the main
branch Shoebot doesn't have that and it becomes rather confusing.

One thing i noticed was that the rendering functions are specified
*inside* the bot.

One of my concerns with the last refactoring of the Shoebot code, months
ago, was to decouple Bots (language front-end) and Canvases (graphics
back-end), via a common object-passing interface -- Grobs. The point of
this is that you could develop alternative graphics backends that could
work with the Nodebox language; in the same way, you could create new
language front-ends (like i did with Drawbot) without fussing around
with the graphics back-end.

However, this new architecture brought new slow-downs since before that,
we had only a Bot that called directly the Cairo context (like your
current approach, if i'm reading this correctly). In the old way,
Nodebox commands were directly translated to context calls; in the new
way, Nodebox commands are translated into Grobs and put into a drawing
queue, which then is read by the Canvas class and finally drawn. This
2-pass approach creates a lot more bottlenecks, since we have to handle
the queue ourselves, whereas before we just delegated that to Cairo.
Stuff is thus slower, but i personally would invest in the new
architecture since those bottlenecks can be profiled and traced down as
we test it.

Just wondering if you took this detail into account? Not to put you off
-- i'm really happy to see this, it's great stuff!

Post by Stuart Axon
Here: http://bitbucket.org/stuaxo/toybot/
At the moment theres just shoebot.py.
If you run it, you'll get 10 SVGs.
The Bot itself is embedded at the end of the code for now.
background(r, g, b)
size()
rect()
FRAME
fill(r, g, b)
1
Implement transforms.
1a
- The transform should be tracked as the bot runs, with just the absolute
coordinates being output to the draw queue at each stage.
http://dev.nodebox.net/browser/nodebox-java/branches/rewrite/src/java/net/nodebox/graphics/Grob.java
getCenteredTransform()
1b.
Write test bots, to check transforms on nodebox work in the same way as here.
2
Get output working to a window.
2a
Implement Either GTK, or use something else for output, e.g. pyglet
2b
Test performance against shoebot classic.
3
Review external API.
- It should be as easy as possible to pass in a Cairo context and get a frame
drawn on it.
- Possibly be able to iterate through frames on a bot.
4
Integrate new code.
4a
Start new shoebot branch
4b
Move code that is working into branch
4c
Stub out other methods with NotImplemented
S++
_______________________________________________
Shoebot-devel mailing list
http://lists.tinkerhouse.net/listinfo.cgi/shoebot-devel-tinkerhouse.net

[Stuart Axon]

Wed, May 5, 2010 6:55:42 PM
Re: [shoebot-devel] Toybot branch up
...
...
View Contact
- I love the callable passing idea
- Great that you used faux-private variables (_variable), the main
branch Shoebot doesn't have that and it becomes rather confusing.
One thing i noticed was that the rendering functions are specified
*inside* the bot.
One of my concerns with the last refactoring of the Shoebot code, months
ago, was to decouple Bots (language front-end) and Canvases (graphics
back-end), via a common object-passing interface -- Grobs. The point of
this is that you could develop alternative graphics backends that could
work with the Nodebox language; in the same way, you could create new
language front-ends (like i did with Drawbot) without fussing around
with the graphics back-end.
However, this new architecture brought new slow-downs since before that,
we had only a Bot that called directly the Cairo context (like your
current approach, if i'm reading this correctly). In the old way,
Nodebox commands were directly translated to context calls; in the new
way, Nodebox commands are translated into Grobs and put into a drawing
queue, which then is read by the Canvas class and finally drawn. This
2-pass approach creates a lot more bottlenecks, since we have to handle
the queue ourselves, whereas before we just delegated that to Cairo.
Stuff is thus slower, but i personally would invest in the new
architecture since those bottlenecks can be profiled and traced down as
we test it.
Just wondering if you took this detail into account? Not to put you off
-- i'm really happy to see this, it's great stuff!

I haven't given these much thought yet.

The idea at this stage is to prove that the main problems are fixable, with
a minimal codebase.

WRT Graphics:
At the moment rendering functions are in
BotImpl (renaming to BotBackend) - Seems the same as Context in nodebox, but I'm not 100% sure?
Grob
BezierPath

I need to give some thought to how a graphics backend might work,
would they be more like cairo or more like shoebot ?

It might help to know what other rendering backends we want.

I want to put this off until after transforms and rendering to a window work.
At that point this'll be ready to be pulled apart and put back together
in a branch of shoebot.


WRT to different grammars:
I was imagining these could be implemented by the Bot classes:
Bot -> Nodebox Strictly the nodebox API
Bot -> Shoebot Our extended API
Is that enough, or will we need different types of BezierPath etc ?

[ricardo lafuente]

Post by Stuart Axon
I haven't given these much thought yet.

Re-reading your code and comments, it's not that big a deal, because as
you say

Post by Stuart Axon
The idea at this stage is to prove that the main problems are fixable, with
a minimal codebase.

...this is the proper way. Even in a worst case scenario where
performance doesn't differ much between trunk and your branch, there's a
lot of alternative approaches here that make sense to be placed into the
main version, which is rather crufty.

Post by Stuart Axon
At the moment rendering functions are in
BotImpl (renaming to BotBackend)

Yes, its purpose becomes clearer that way.

- Seems the same as Context in nodebox, but I'm not 100% sure?

I believe so -- actually, Bot/Canvas in Shoebot == Context/Canvas in
Nodebox.

Post by Stuart Axon
I need to give some thought to how a graphics backend might work,
would they be more like cairo or more like shoebot ?

Well, the Cairo way of drawing is very close to Nodebox/Shoebot (drawing
stack, transforms, etc.). However, other backends that could be
implemented in the future (i remember you working on 3D) might not be so
close; hence the choice to have our own drawing stack and elements,
which can then be passed on to a backend.

Post by Stuart Axon
It might help to know what other rendering backends we want.

OpenGL (Pyglet/GtkGLExt) seems like the next target. Once stuff is
stabilised, i'd also like to take a stab at implementing a Qt/Arthur
backend, which apparently can be faster than Cairo in some cases.

Post by Stuart Axon
I want to put this off until after transforms and rendering to a window work.

Agreed, stuff should work first

Post by Stuart Axon
At that point this'll be ready to be pulled apart and put back together
in a branch of shoebot.

Sounds like a good plan.

Post by Stuart Axon
Bot -> Nodebox Strictly the nodebox API
Bot -> Shoebot Our extended API
Is that enough, or will we need different types of BezierPath etc ?

Well, these are 2 different things:
- Bots: Nodebox and Shoebot (= extended Nodebox) are the ones we have
now, plus Drawbot. Another grammar that i'd like to try would be a
Pythonic version of Processing.
- BezierPath (and Grobs in general) are the 'agnostic' representations
of drawing elements, so that any Bot will create grobs, and any
canvas/rendering backend can read them and output image. So i think
we're good with the classes that we have already.

[Stuart Axon]

- Bots: Nodebox and Shoebot (= extended Nodebox) are the ones we have now, plus Drawbot. Another grammar that i'd like to try would be a Pythonic version of Processing.
- BezierPath (and Grobs in general) are the 'agnostic' representations of drawing elements, so that any Bot will create grobs, and any canvas/rendering backend can read them and output image. So i think we're good with the classes that we have already.

Cool, I do want to be able to extend BezierPath in the Shoebot case though, and if possible not when working under Nodebox.
Although, maybe extending it in all cases doesn't matter, it would be nice if the nodebox implementation was as authentic
as possible.

[Dave Crossland]

This 2-pass approach creates a lot more bottlenecks, since we have to handle
the queue ourselves, whereas before we just delegated that to Cairo.

Random thought, probably useless: Could using Kamaelia to handle the
queue help speed things up?

[ricardo lafuente]

Post by Dave Crossland

This 2-pass approach creates a lot more bottlenecks, since we have to handle
the queue ourselves, whereas before we just delegated that to Cairo.

Random thought, probably useless: Could using Kamaelia to handle the
queue help speed things up?

Good suggestion -- from the Kamaelia website ():

"In Kamaelia you build systems from simple components that talk to each
other. This speeds development, massively aids maintenance and also
means you build naturally concurrent software. It's intended to be
accessible by any developer, including novices. It also makes it fun :)

What sort of systems? Network servers, clients, desktop applications,
*pygame based games*, transcode systems and pipelines, digital TV
systems, spam eradicators, teaching tools, and a fair amount more :) "

If it's good for PyGame applications, i'm sure it's good for us. TBH,
i've never got to fully understand the Kamaelia concepts of concurrency;
but integrating Kamaelia would give us the benefit of
parallel/multi-threads without having to dive into the mud of Python
threads.

The 'Getting started' section of the website
(http://www.kamaelia.org/GetKamaelia.html) explains the base concepts
and how to develop with Kamaelia. If there's anyone interested in taking
this on together, we should definitely stage an IRC sprint :-)

[Dave Crossland]

Kamaelia would give us the benefit of parallel/multi-threads without having
to dive into the mud of Python threads.

That was what I was really thinking, yes.

[Stuart Axon]

Sent: Thu, May 6, 2010 10:23:53 AM
Subject: Re: [shoebot-devel] Toybot branch up
On 5 May 2010 18:55,
ricardo lafuente<

Post by ricardo lafuente
This

2-pass approach creates a lot more bottlenecks, since we have to handle

Post by ricardo lafuente
the queue ourselves, whereas before we just delegated that to

Cairo.
Random thought, probably useless: Could using Kamaelia
to handle the
queue help speed things up?

Good suggestion -- from
"In Kamaelia you build systems from simple

components that talk to each other. This speeds development, massively aids
maintenance and also means you build naturally concurrent software. It's
intended to be accessible by any developer, including novices. It also makes it
fun :)

What sort of systems? Network servers, clients, desktop

applications, *pygame based games*, transcode systems and pipelines, digital TV
systems, spam eradicators, teaching tools, and a fair amount more :) "

If

it's good for PyGame applications, i'm sure it's good for us. TBH, i've never
got to fully understand the Kamaelia concepts of concurrency; but integrating
Kamaelia would give us the benefit of parallel/multi-threads without having to
dive into the mud of Python threads.

The 'Getting started' section of the

website (http://www.kamaelia.org/GetKamaelia.html) explains the base concepts
and how to develop with Kamaelia. If there's anyone interested in taking this on
together, we should definitely stage an IRC sprint
:-)

I've been building small test programs with cairo to test stuff before putting it
into shoebot, it might be worth building one for this.


We need to know if we can pass two things between threads: cairo contexts and
function references.

If we can do both of those, and it works quickly then we can multithread shoebot.


If you can get something like the following working, we can use it in shoebot:

Bot Thread - adds draw functions to a drawqueue (a list)

Render Thread 1 - Accepts a list of functions and renders them to a cairo surface (we could just say an ImageSurface for now)
Render Thread 2 - another render thread

Output Thread: - Accept output from render threads (in order) and draw to a window somehow.

[Stuart Axon]

Post by Dave Crossland
Random
thought, probably useless: Could using Kamaelia to handle the

queue help

Post by Dave Crossland
speed things
up?
From a very brief look Kamaelia seems to be about concurrency, it might be good for having seperate bot/render threads.

For the moment I'll try and keep things simple though (although it's def worth thinking about these things) :)

[Dave Crossland]

Post by Dave Crossland

Could using Kamaelia to handle the queue help speed things up?

From a very brief look Kamaelia seems to be about concurrency,

Yes

Post by Dave Crossland
it might be good for having seperate bot/render threads.

Its pure python, and meant to be very fast, and while its meant for
concurrent systems, it might help with latency issues even on a
non-concurrent queue.....

Post by Dave Crossland
For the moment I'll try and keep things simple though (although it's
def worth thinking about these things)  :)

Sure thing :-) Kamaelia is fairly simple, though.