async

This page is obsolete. Features described in this page have been implemented by haxe-continuation, with a few syntax changes.

Here's a topic brought here through this ML discussion:
http://haxe.1354130.n2.nabble.com/NodeJs-is-a-big-opportunity-for-Haxe-tt6892913.html

The big idea is that we have a very interesting use case with nodeJs of totally asynchronous existing APIs.
So as we start using them, we think we can go further than current Js programmers if we provide high level transformations of the control flow in order to provide some easy way to work within this concurrent setting.

The aim of this page is to summarize discussions around this subject as Nicolas is willing to rethink about it as since the 2.08 such transformations could be more easily implemented.

Note that this kind of new possibility is not tight to nodeJs and could be used for a broader usage cases.

Those transformations are related to Async programming, Continuation Passing Style (CPS), Delimited continuations, Monads, ..
You may google about it.

Some things we want to be able to achieve with them:

- preserve a syntax close to existing haxe.
- be able to make several async execution in parallel (and not only nested ones).
- be able to manipulate those (represent them with a type for deriving combinators ?) ??

-------------------------------------------------------------------------------------------------------------------------------

Rough idea:

Ex 1:
function foo(x : Int, cb : Int -> Void) : Void

var a = foo(5).async(); // this call tranform the code after this call into a function (a being his parameter)
// and feeds it as the last argument of the foo function
trace(a); // so that you can use it!

Note:
Some nodeJs function have their callback defined at the begining of the arguments (not very common).
In addition, various overloads may add to the noise..
an alternative syntax would help with that:
var a = foo(5, _);
var b = foo(_, 5);
// error????????? var c = bar (_, 5, _);

Ex 2:
function factorialOf(x : Int, cb : Int -> Void) : Void
function stringLength(x : String, cb : Int -> Void) : Void

var a = factorialOf(100).async();
var b = stringLength("toto").async(); // this call should not be deferred after the previous callback call
// this is not compatible with Ex1, or it would requiers some more involved
// analysis to find the right place and how o combine them
return a + b;

Ex 3:
// most nodeJs function callback have more than one parameter:
function connect(url : String, cb : Error -> Socket -> Void) : Void;

// if we consider the previous example; it would need some kind of indirection:

var a = factorialOf(100).async();
var b = stringLength("toto").async();
return a.??? + b.???;

-------------------------------------------------------------------------------------------------------------------------------

I think we may better separate the call of the function and the creation of the callback to feed it in.

the above sample would be:
var contA = factorialOf(100, _);
var contB = stringLength("toto", _);

then creating the function to feed in could be done this way:

[ a = contA, b = contB].async();
// here starts the callback (a mix of two functions) called when both contA and contB call their respectives callbacks.
// a and b should each contains all the parameters of their functions.

// in the more general case the array is dynamic, we would need to get an array of the parameters (typed based on the parameters unification).
var arr = [contA, contB];
var values = arr.async();
etc..

// not to forgot that doing that is still interesting (to share
var data = contA.async()
.. // big computation
var elem = allElements.async() // called 100 times (one call per elem)
.. // small computation

// we have a choice; either if the calls happend at the creation of contA and contB or when calling async..

On the semantic side; There's a fundamental problem if we want to mix several continuations into one,
Indeed, a continuation (or a callback) may be called 0, 1 or several times (for each elements of a collection) in a function body, so it is not clear if what we're willing to achieve is enforcable at the general continuation level; it would requiers some sort of garantees (>0..) in order be able to combine several continuations (It's a choice).

Sticking to the real nature of what could be done, we may see the callback calls as reactive events. The semantic associated with single / multiple calls is easier as one can choose how to handle that for his specific needs.
The downside is that this additional control can be hard to understand and it is becoming difficult to provide garantees about it.. (note that this will remain true for all extern code)
(beware of leaks when implementing it..)

In the end, either solution we want to go; there are some tools we may consider independantly to shape ideas:

The ability to easily curry a function (like callback today but at various places).
the ability to combine several functions into one with each input parameters present into the final one (and the associated 'magic' which performs the final call when all is bound (+semantic issues we talked earlier) ).
(beware of the case one want to perform that on an array of functions.
(( derived from the above:
The ability to name anonymous function parameters to reference them (useful when mixing functions) - maybe this is closer to the convergence between tuples and method parameters.
))
The ability to transform the rest of function as a function with named parameters (Or if we want to get continuations returning values to compose them, the ability to define a delimited function - whos returning value is the value of it's delimitation (reset in shift/reset nomenclature).

Perhaps I think to generally or do not think enough out of the box.. these are my thoughs today.

-------------------------------------------

hello, I see two issues with concurrency or async.
one is dependency, the other is a shared resource.
imagine that every expression was async, but one variable is still dependent on results from another previous variable.
another issue is shared resources that have to be locked or operated on atomically.
have you seen scoop from eiffel?
http://docs.eiffel.com/book/solutions/concurrent-eiffel-scoop
it is a simple solution with minimal api for the users.
maybe the compiler can figure out some things itself? which statement can be async and which can't?
that would relieve the users, but almost changes the entire language... some thoughts..

version #15592, modified 2012-10-27 11:36:00 by Atry