Fibers

One of the most notable differences between Node.js and other languages is Node’s non-blocking IO, and resulting asynchronous programming model. While this has many advantages, it can often be challenging to work with complex callback structures – a phenomenon often termed “callback hell” or “pyramid of doom.”

Many different libraries and techniques exist to help programmers manage this complexity. While any can be used with Carbon.io, as it supports both synchronous and asynchronous programming styles, Carbon.io’s preferred technique is to use fibers.

While fibers provides basic coroutine support, Carbon.io implements the abstractions necessary to allow you as the programmer to more easily write non-blocking code in a synchronous style.

Consider this standard asynchronous operation:

1
2
3
4
5
6
7
fs.readFile(path, function(err, data) {
  if (err) {
    console.log(err)
  } else {
    console.log(data)
  }
})

With fibers you can do this:

1
2
3
4
5
6
7
8
__(function() {
  try {
    var data = fs.readFile.sync(path)
    console.log(data)
  } catch (err) {
    console.log(err)
  } 
})

Note that no callback is passed to readFile. Instead readFile evaluates to its result or throws an Error.

Creating Fibers

In order to synchronously call asynchronous functions, the calls must execute inside of a Fiber.

This is done via the __ operator. The __ operator takes a function of zero arguments and executes that function inside of a Fiber. Calls to functions via sync inside the Fiber will cause the Fiber to block until the function’s callback is called, but will do so without blocking the process. Any IO performed inside those functions is still asynchronous.

Here is a full example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
var fs = require('fs')

var carbon = require('@carbon-io/carbon-io')
var __ = carbon.fibers.__(module)

__(function() {
  var path = process.env.EXAMPLE_PATH || '/foo/bar'
  try {
    var data = fs.readFile.sync(path)
    console.log(data)
  } catch (err) {
    console.log(err)
  } 
})

Again notice that no callback is passed to readFile.sync.

The __ operator itself executes synchronously or asynchronously depending on the context in which it is invoked. If it is called within the context of an active Fiber, it executes synchronously, otherwise it executes asynchronously. This allows you to have multiple entry points to your application, bootstrapping as appropriate, while pulling in other Fiber aware modules synchronously. While the usefulness of this may not be immediately obvious, it should become clear as you become more familiar with the Carbon.io landscape (see: @carbon-io/test-tube).

To illustate, the following code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
var fs = require('fs')

var carbon = require('@carbon-io/carbon-io')
var __ = carbon.fibers.__(module)

console.log('foo')

__(function() {                 // asynchronous
  
  console.log('---')

  __(function() {               // synchronous
    console.log(0)              
  })
  
  __(function() {               // synchronous
    return 1
  }, function(err, result) {
    console.log(result)
  })

  __(function() {               // synchronous
    console.log(2)
  })

  __(function() {               // synchronous
    console.log(3)
  })

  __(function() {               // synchronous
    console.log(4)
    return 5
  }, function(err, result) {
    console.log(result)
  })
})

console.log('bar')

__(function() {                 // asynchronous

  console.log('---')

  __.spawn(function() {         // asynchronous
    console.log(0)              
  })

  __(function() {               // synchronous
    return 1
  }, function(err, result) {
    console.log(result)
  })

  __.spawn(function() {         // asynchronous
    console.log(2)
  })

  __(function() {               // synchronous
    console.log(3)
  })

  __(function() {               // synchronous
    console.log(4)
    return 5
  }, function(err, result) {
    console.log(result)
  })
})

console.log('baz')

produces this output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
foo
bar
baz
---
0
1
2
3
4
5
---
1
3
4
5
0
2

In addition to the bare __ operator, as you may have noticed from the previous example, fibers provides the __.spawn function which will always execute asynchronously (__ actually delegates execution to __.spawn in the case that it is invoked outside of an active Fiber).

Using sync

The fibers module exposes a sync property on both Functions and Objects.

When sync is used on a function, sync returns a new function that is a synchronous version of the original. The new function takes the same arguments as the original, except the final callback parameter, and returns the result or throws an Error if an error occurs.

1
2
3
4
5
try {
  var data = fs.readFile.sync(path)
} catch (err) {
  console.log(err)
}

When sync is used on an object, sync returns a new object that is the same as the original except all methods are synchronous.

1
2
3
4
5
try {
  var data = fs.sync.readFile(path)
} catch (err) {
  console.log(err)
}

When working with instances of “classes” where methods may interact with a this, one should use the second form: obj.sync.<method>(...).

Return Values

Currently, __ and __.spawn return undefined. If you want to perform some computation asynchronously and retrieve the result, you can do this by passing an “errback”.

Using Fibers in Carbon.io applications

When using Fibers in Carbon.io applications you will want to make sure that all entry points to the event loop are wrapped in __.

One example of this is your main program. You will often see example applications structured like this:

1
2
3
__(function() {
  // application code goes here
})

This ensures that your application code is run in a Fiber, which will allow you to use sync in your application code.

Additionally, you will often see test modules structured as follows:

1
2
3
4
5
__(function() {
  module.exports = o({
    // application code and exported functionality
  })
})

This indicates that the module can be used as both an entry point and as a dependency. Dependants of modules written in this fashion should be careful to require these dependencies from within a call to __. If this pattern is not adhered to, strange behavior will result as module.exports will be initialized asynchronously and the dependant module will receive an empty object (e.g. {}) instead of the expected value.

Carbon.io Service objects automatically wrap each HTTP request in a Fiber so that your code that handles HTTP requests can use sync.

You may find that there are other times when you need your code to run in a Fiber, such as the callback to a timer. In each instance you can use the __ function to create and run a Fiber.

The application structure section of the server guide provides more information on how to use Fibers with Carbond.

Debugging

If your application is exiting suddenly or behaving differently than you would expect, it could be that an unhandled exception is the cause. Normally, these exceptions are swallowed silently (note, this is a programming error as these exceptions should be handled further up the call chain), but you can enable logging of these errors via the debug module by adding DEBUG="@carbon-io/fibers" to the environment.