muSE - muvee Symbolic Expressions

Saying "hello world!" on MacOSX ..

2010-11-29T11:13:00.004+08:00

Here is a different "hello world!" by muSE on MacOSX :)


; Load the AppKit framework containing the NSSpeechSynthesizer class
[[NSBundle bundleWithPath: "/System/Library/Frameworks/AppKit.framework"] 
   load]

; Initialize the "hysterical" voice and say "Hello world!" with it.
(define hysterical 
  [[[NSSpeechSynthesizer alloc] initWithVoice: "com.apple.speech.synthesis.voice.Hysterical"] 
    autorelease])

[hysterical startSpeakingString: "Hello world!"]

r580 release

2009-07-12T17:23:00.003+08:00

Alright! r580 it is! ... just 'cuz I have to say STOP! at some point, actually.

All things considered, r580 should be a more usable and more robust release than most earlier ones. Apart from the usual features and bug fixes, the examples have also been updated to this release and placed in the trunk folder along with the source code for added convenience.

For details, see the AboutRelease580 wiki page.

Candidate for next release

2009-07-08T10:11:00.005+08:00

r535 is a likely candidate for the next binary build. It is also the version that I hope to synchronize with muvee's own use of muSE in muvee Reveal in the upcoming mid-August update. I've already updated the generated documentation to reflect r535.

So far, its looking good and r535 solves many of the problems that existed before. I'm finally at the point including the and it (see TheAndIt) in the language. Need to iron out a few kinks regarding them in conjunction with finally before I can say I'm done with that feature. So there'll surely be at least one more tag after r535. Also, I haven't tested on linux just yet - only macosx and win32.

Announcing muvee-style-authoring public beta

2009-04-17T10:14:00.005+08:00

What a long interval from the previous post, but I'm very excited about this one and have waited a long time for it.

As I've claimed earlier, both muvee autoProducer 6.x and muvee Reveal use muSE as the core language for their styles, but told you nothing about how to create these styles yourself. Now we at muvee are opening up everything you need to create your own styles.

Project page: muvee-style-authoring

We've put up lots of documentation, a few tutorials to get you started (the set will grow), and have a helper tool to work with the styles you create. So go ahead, play around and give us feedback. If you get the latest version of muvee Reveal, all the code for the built-in styles is open for you to explore and copy for your own styles.

Note that the documentation and examples are all about muvee Reveal - i.e. you can only create your own styles for muvee Reveal.

Experimental 'the' and 'it' primitives

2008-07-19T11:15:00.001+08:00

Version 437 introduces a couple of experimental primitives the and it aimed at minimising the need for local variables created to hold intermediate results.

Main page muvee-symbolic-expressions

Description on the wiki page TheAndIt

Experimental win32 executable - muse-v437-experimental-win32.zip

Blogged with the Flock Browser

muvee Reveal

2008-06-12T13:44:00.001+08:00

muvee Reveal - the latest incarnation of muvee's flagship product - has just been released on 11 June 2008! The behaviours of the 8 bundled styles are specified using muSE, in addition to all the styles developed for the now discontinued muvee autoProducer 6.1.

muSE v372 ...

2007-12-29T10:59:00.000+08:00

Download - for macosx, for win32.
Summary of changes between v343 and v372 -

Features: Simple module support, more flexible xml reader, polymorphic get and put.
APIs: memports, muse_read_xml_node and muse_raise_error
Bug fixes: GC fixes, muse_pread() and anonymous symbols.

Detailed changes -

Features

Support for simple modules.
The xml reader has been made slightly flexible in dealing with non-standard input. In xml tag attributes, attrib="value" and attrib='value' are standard but attrib=value is not. Since a lot of html code uses attrib=value form, this is also now accepted by the parser. Note that this is still not enough to accept old html files because they might contain non-xhtml tags such as <br> which should be <br/> in xhtml.
Added a notion of a 'prop' view for getting and setting properties of arbitrary native objects. Properties are accessed and modified using (get obj prop) and (put obj prop value). i.e. get and put are now polymorphic over all object types that support the 'prop' view. See muse_prop_view_t struct for the interface.
Added new "box" type in the same spirit as R5RS boxes.
A symbol of the form ".name" will be expanded into the expression (-> self 'name) at read time. This makes it easy to refer to object properties from within method bodies.
Made module names behave like "do", except that within the scope of the module expression's body, symbols are interpreted in the scope of the module - irrespective
of whether the module expression is at the global level or within a closure creating expression.
This feature is made possible by the introduction of a new "scope view" - id 'scop' - which lets an object provide a hook into the closure creation code to introduce local bindings that aren't visible in the s-expression itself.
The scope view lets us define context sensitive forms. Ultimately, all binding introducing forms - define, let, case, fn, fn: and gfn - can be implemented by providing a different scope view for each so that there is a constant testing overhead during closure creation instead of depending on the number of such special forms that need to be supported.

New APIs

Memport objects can be created from the C api.
muse_read_xml_node is now exposed in the C api.
Added muse_raise_error() for raising muSE exceptions in native code.

Bug fixes

(major): The main process structure was being prematurely destroyed before the gc function call returned. As a consequence, the area around the process's stack got corrupted and muse_destroy_env() won't exit cleanly. This went undetected because the primary use was via the repl which can be exited only using (exit) which terminates the whole process.
(major): Major GC bug that crept in from a process related tweak to make the gc run atomically. Ever since the tweak (to wrap muse_gc_impl() within an atomic block) the gc never ran but simply grew the heap. FIXED! Whew! (I'd been wondering for a while why a gc never got triggered and the heap was grown instead during several large data set experiments, but didn't probe further. Lesson learnt - pay attention to your hunches!)
(minor): muse_pread() wouldn't return EOF if there were only spaces or comments before end of file (or port in general). Modified it so that if only spaces and/or comments are found before it hits EOF, it dutifully fails.
* (minor): Anonymous symbols cannot have the sym itself as the head value any more due to process architecture. So changed the head value of anonymous symbols to () (or MUSE_NIL). Since this affects the gensym operator which converts an anonymous symbol into a permanently interned symbol, muse_intern_symbol was also modified to change the head value of an interned anonymous symbol to refer to the proper symbol location in the symbol vectors of all processes.

Other changes

Native objects are no longer evaluated lazily when they appear in the function position. They are treated like any other native function. This allows us to define native objects which evaluate their arguments inside a special kind of context.
To compensate for this loss, the "lazy" operator's definition has been modified (and simplified) to cause a function application (native or user defined) to be
postponed until forced. Using the new "lazy" operator, you can use tail recursion with native functions too.

New v338 of muSE available

2007-11-03T17:06:00.001+08:00

Note that v338 has been replaced by v343, which contains a minor fix to the exception handling code. Prior to the introduction of finally, an expresion such as (try (raise 'error)) probably indicated a programming error since no handlers were specified. With finally however, a try without handlers is valid since it may install finalizers using finally.

Downloads page.
Summary of changes between v285 and v338 -

Language and library features

mickey processing support - turns muSE into a scheme-based macro expander.
Added native implementations of reverse and reverse! functions.
with-connection-to-server has been replaced with open.
Added split function to split a string at given separator characters.
[MAJOR!] Added (finally ...) to the exception mechanism. See ExceptionHandling.
Invocation of spawn has been changed.

Bugfixes and enhancements

muSE can now be built as a dll.
Bug fixes in hashtable object.
Fixes to lazy evaluation mechanism. Incorrect behaviour in certain obscure circumstances.
Fixes to networking functions - particularly read will now return () when the socket it is reading from is closed at the other end.
Some fixes to exception handling implementation.
list-files now usable under unix using same (but limited) wild-card syntax like windows.

Finally!

2007-10-20T01:18:00.000+08:00

A while ago, I'd written about raising and handling exceptions in muSE. The key idea, borrowed from Common Lisp, is that when an exception is raised, the execution context should not be unwound to the point where the exception can be handled. This allows an exception handler to resume execution from the point the exception occurred or try another handler, starting again from the inner most context. The current try-raise-retry facility in muSE manages to capture this requirement fairly well. However, there was one glaring omission - the inability to specify scope limited cleanup operations. In v332, muSE gets a cleanup mechanism ... finally ... I mean literally a finally block .. and this post is all about it. If you're going "oh yeah I know all about finally, Java has it", I humbly urge you to read on. You'll wish you had this in Java.

In muSE, you can now place a code in a finally block that can feature anywhere within an expression protected by a try. "Anywhere" doesn't just mean the visible lexical scope, but the entire execution scope. Not only can you have finally anywhere, you can have as many of it as you need - you can even accumulate finalizing actions in a loop. The finally block gets captured into a thunk that gets evaluated at the end of the scope of the try context in which the block occurs. The thunks execute in the reverse order in which they are created. The thunks can refer to any variable in its lexical context, since a closure is created when evaluating a finally block.

Here is a simple example that reads a number from file F, multiples it by a given value and returns the result.


; Raises an exception if the file couldn't be opened.
(define (safe-open file)
  (let ((f (open-file file 'for-reading)))
    (if (eof? f)
      (raise 'FileOpenError file)
      (do (finally (print "Closing file" file) (close f))
          f))))

; Read a number from a file
(define (read-number file)
  (define F (safe-open file))
    (let ((n (read F)))
      (if (number? n) n (raise 'NotANumber n))))

Here is a way to read a number from "F.scm" and if you don't get a number, use a default value -


(try (read-number "F.scm")
  (fn (ex 'NotANumber n)
    42))

Here is a way to read a number from "F.scm" and if the file doesn't exist, use "Default.scm" instead -


(try (read-number "F.scm")
  (fn (cont 'FileOpenError file)
    (cont (safe-open "Default.scm"))))

When do exceptional conditions occur?

In pure computations, they can happen when an unsupported - "unexpected" - input value is encountered when calculating the result. Let us call these p-type exceptions. The other type is when dealing with side-effecting computations, of which IO is arguably the most dominant case. Let us call these s-type exceptions.

p-type exceptions can be dealt with in a language which allows one to substitute another expression for one containing the exceptional case, when the case does happen. muSE does this fairly well with its try-raise-retry operators. The example function in the earlier article shows how a pure function with an exceptional condition can be tamed by the calling context, given an expression substitution mechanism.

s-type exceptions cannot be dealt with adequately with a pure-expression substitution mechanism. In the interest of composability, it is often desirable to wrap side-effecting expressions into what feels like a pure function - for example, a data structure lookup that appears to never fail might be implemented via a connection to a remote database. To do that, we need to be able to execute one or more side-effecting cleanup operation, such as closing the database connection, before the wrapper function returns to its calling context.

With the new finally block, I believes muSE satisfies both these cases.

muSE on MacOSX Intel

2007-10-01T11:55:00.000+08:00

Now I have an Intel MacOSX machine and can build and verify muSE on it.
The v285 build was broken on OSX Intel machines and I've fixed it (Issue 32).

Here is a build with the fixes - muSE-v316-objc-macosx.tar.gz"

Curry Howard isomorphism

2007-08-15T07:31:00.000+08:00

Curry Howard isomorphism states that every logical proposition corresponds to a type. (See also Curry-Howard-Lambek correspondence.) I've been fascinated by this recently, in the context of Haskell, 'cos it yields very interesting and powerful functions. Often you can just take the type expression in Haskell and write a function that has the type.

For example, consider the trivial logic "theorem" -

If it is true that proposition A implies proposition B, then A is provable implies B is provable.

If you notate provability by the symbol #, you can write that as -

(A => B) => (#A => #B)

If you consider => to be the function type constructor (in Haskell), this gives you the type of fmap where # is any functor. It looks like you can uniformly map "is provable" to some kind of containment - like a list or a set.

Here's a very interesting function derived from another logic proposition - From Löb's Theorem to Spreadsheet Evaluation.

Here's an implementation of the loeb function in muSE -


; Limitation - fmap over lists only
(define (fmap f x) 
    (if x 
        (lcons (f (first x)) 
               (fmap f (rest x))) 
        ()))
(define (loeb x) (fmap (fn (a) (a (loeb x))) x))

.. and a variant on the author's example -


(define test
    (list   (fn (x) 17)
            (fn (x) (apply + (drop 3 x)))
            (fn (x) (* 2 (nth 3 x)))
            (fn (x) (+ (nth 4 x) (nth 5 x)))
            (fn (x) 3)
            (fn (x) (nth 0 x))))

The stair climbing robot

2007-08-15T07:19:00.000+08:00

An interesting problem called "the stair climbing robot" was recently posted on LtU.

Your stair-climbing robot has a very simple low-level API: the "step" function takes no argument and attempts to climb one step as a side effect. Unfortunately, sometimes the attempt fails and the robot clumsily falls one step instead. The "step" function detects what happens and returns a boolean flag: true on success, false on failure. Write a function "step_up" that climbs one step up (by repeating "step" attempts if necessary). Assume that the robot is not already at the top of the stairs, and neither does it ever reach the bottom of the stairs. How small can you make "step_up"? Can you avoid using variables (even immutable ones) and numbers?

I thought I'll try to solve it using muSE's lcons lazy function. Here's a solution -


; A step function which fails 50% of the time.
(define (step) 
    (if (= 0 (rand 2)) 
        (do (print "DOWN") ()) 
        (do (print "UP") T)))
 
; Gets the second element of a list s,
; forcing the value of the first element if
; the list happens to be lazy.       
(define (forced-next s) (first s) (first (rest s)))

; (steps) generates a sequence of
; successful upward steps. To climb N steps,
; just take N elements from (steps).
(define (steps)
    (lcons (if (step)
                T
                (forced-next (rest (steps))))
            (steps)))

; Climb 1 step
(take 1 (steps))

spawn behaviour changed in v295

2007-07-31T21:12:00.000+08:00

The spawn function has different semantics from v295 of muSE.

Prior to v295, spawn took a thunk and evaluated it in a separate process repeatedly until the thunk evaluated to a non-() value. This was done to make it simpler to write server processes that run the same code over and over until some condition occurs that makes it terminate.

Since v250, muSE has had support for stack optimized tail recursion and lazy evaluation. This means we can now write server processes as tail recursive functions that never terminate. Therefore we don't need the extra help provided by the spawn function any more.

From v295, (spawn thunk) will cause (thunk) to be evaluated only once and it will discard the result.

Expressing server processes as tail recursive functions, besides stating the obvious, gives you new ways to modularize parts of the server. If your server needs to be placed into modes when it receives certain input, each of these modes can be expressed as tail recursive functions that revert back to the main server function whenever an exit mode message is received.

XML processing

2007-07-17T02:15:00.000+08:00

muSE now (in v285) has support for parsing and generating simple XML structures. See -

read-xml and write-xml
xml-v271.scm for useful predicates and combinators
XMLProcessing wiki page

Getting lazy

2007-05-23T04:30:00.000+08:00

Recently, in v250 of the muSE core library, I added support for stack optimization of tail calls - finally!

muSE has been missing the ability to do tail recursion for ever and now its finally there. The absence of tail recursion support had a rather ugly but practical consequence for the language - the while and for looping constructs had a reason to exist because you couldn't loop otherwise without blowing the stack top. I'm glad to note that they can now be safely RIP-d.

Tail call optimization as implemented does not apply to tail calls of native functions. It only applies to applications of user defined functions that appear in the tail position in a block.

As a by-product of the tail call optimization, I was surprised to find it easy to implement lazy evaluation. There's an experimental primitive lcons which can be used to construct a list lazily. This operator is inspired by Dan Friedman's TR44: CONS should not evaluate its arguments technical report. Some classic examples can be found in examples/lcons-v238.scm in the source tree.

Exceptions redux

2007-03-30T21:28:00.000+08:00

(Ref to muSE v234)
Although I've written about it a bit earlier, I did not do muSE's version of exception handling and recovery enough justice then. I simply described how to use it, that too incompletely.

The bigger picture here is the question -

What interface does a function exactly provide its user?

In languages such as C, functions as computation specifications are used only via their arguments, or via side effects on the environment. A function's documentation will typically tell you what it will do when you pass it certain arguments, what its "return value" will be and what side effect, if any, it will have on the environment at the time it is called. No facility was provided in the language to deal with exceptional conditions. Programmers usually used the "return value" to flag error conditions, which are expected to be "handled" by the caller, usually propagating the condition up the stack until something can be done about it.

C++ introduced (let's assume for simplicity) into the mainstream programmer's consciousness the notion that a function might "throw" an "exception" rather than return a value. The "throw" construct indicated that the function's computation was being terminated and it was not going to return to its caller as expected to do so under normal conditions. Instead, it was going to bounce around into "catch" blocks higher and higher in the stack of calls until one block can "handle" the exception - with the hope that it will somehow be able to recover from it. The stack would be unwound until such a willing handler is found or the program is terminated.

Java dutifully formalized this mechanism and declared that a function's signature is not only about its arguments and the return value, but is also about the exceptions that it can throw to its caller.

To summarize, a function can either return to its caller normally, or throw an exception. In both Java and C++, once a function terminates either by returning or by throwing an exception, all information about the intermediate state of the computation that was being performed when the exceptional condition occurred is destroyed. C++'s stack unwinding mechanism calls object destructors along the way and Java, since it doesn't keep objects on the stack, provides a finally block where you can do your own cleanup before the exception is passed to the caller.

Hmmm ... now, what can you really do in a catch block in C++ or Java? It is a good exercise to scour the net for C++ and Java code examples that purport to teach the reader the exception mechanism in these languages. I mean do that right now. See if you come up with anything more sophisticated than -


try {
  // Call function which throws exception ..
  int hex = 355, spell = 113;
  abracadabra( hex/(double)spell, "hocus pocus" );
} catch ( IncantationFailedException &ex ) {
  // Handle the exception
}

or maybe


try {
  ...
} catch ( IncantationFailedException &ex ) {
  std::cerr << "Incantation 'abracadabra' didn't work!\n";
}

Even in Herb Sutter's insight filled book Exceptional C++, the pages 25 to 68 on Exception Safety : Issues and Techniques fail to cite a single useful case for the catch clause, let alone recommend typical usage. Though it can be argued that the chapter is on exception safety and not exception handling, all I'm asking for is one good example. If you do find a good case or description of "recommended practice" anywhere, I'd appreciate links as comments to this post.

Is it then a consequence of instructional failure that decent bodies of C++ code hardly ever use the catch mechanism and only use the destructor calls that happen when the stack unwinds? I dare a conjecture here -

The reason catch blocks that do something more substantial than code like the above are hard to find is that the catch mechanism is impotent and consequently nearly useless as a design tool.

The Common Lisp folks have provided a more flexible definition of a function - that its not just about the arguments and the "return value", but can be an arbitrary protocol in which control can pass between the caller and a particular invocation multiple times before the task is done. Peter Seibel talks in detail about the Common Lisp conditions facility in Beyond Exception Handling : Conditions and Restarts and provides a concrete example where a function is useful in more than one context because it exposes such a protocol to its caller.

Having experienced deep frustration on several occasions along the lines of - "if only this function would let me customize the way it treats this condition differently, I won't have to write my own", I realized that the exception mechanism was standing in the way of a significant amount of code reusability. Library functions were deciding too much for me. It is cumbersome to keep passing protocol handlers via arguments for every function, so nobody does that either.

I use muSE as a platform for experimenting with the language features I'd like to have, so I decided to try something more flexible than C++/Java and less cumbersome to use than CL - a mechanism with which you can ignore exceptions when you call a function or go all out to customize the way a function should behave - the former case not requiring any additional work on your part, as it should be. Along the way, I figured that pattern matching dispatch works very well to achieve this goal and makes the handling of exceptions and restarts much simpler than the Common Lisp approach.

Syntactically, the muSE approach is similar to C++/Java - an expression that might not evaluate successfully is wrapped into a try expression and a set of handlers are provided to try when the expression failed to evaluate normally. In order to signal an abnormal condition, a function uses the raise function, supplying it an arbitrary set of arguments that get passed on to the handlers. Here's a simple, though contrived, example (*)-


(define H (fn (x y)
             (try (/ (+ x y)
                     (if (= x y)
                         (raise 'DivideByZeroDiff x y)
                         (- x y)))
               (fn ('ForceResult r) r))))

The above function tries to compute the value (x + y)/(x - y). It can't perform the division if x = y, so it raises a 'DivideByZeroDiff condition in that case. It also provides a code path using which you can force the result of the function to be some arbitrary value in case nothing else can be done about it. Here's a caller trying to compute something using this function (*) -


(define G (fn (x1 y1 x2 y2)
             (try (/ (H x1 y1) (H x2 y2))
               (fn (ex 'DivideByZeroDiff x y)
                  (if (= (- x1 y1) (- x2 y2))
                      (ex 1)
                      (retry 'ForceResult 2000))))))

G is trying to compute H(x1,y1)/H(x2,y2). If one of the differences (x1-y1) and (x2-y2) becomes small, H will raise the 'DivideByZeroDiff exception. Now, G knows that if the two differences are equal, the actual value of the difference doesn't matter to the end result. The H function cannot possibly know this. So in this case, G simply asks H to assume a value of 1 instead of the (raise ...) expression and continue to compute the result. If this is not the case, G decides to cap the value of H to 2000 using the 'ForceResult code path.

Note that H embodies no knowledge about the domain that G knows about. This makes it possible to use H in situations where the original author cannot possibly know how to handle the divide by zero case. The computations performed by this example are trivial, but when they are substantial, the facility to interact with the execution flow of a function definitely renders it more useful in a broader set of contexts and therefore useful as a design tool. Peter Seibel's more substantial log file parsing example can be expressed using the try-raise-retry constructs too.

We've used pattern matching throughout to catch raised conditions and to pass on control to restart points. The Common Lisp approach has equivalent functionality, but to me appears more verbose than really needs to be. In muSE, a function's interface consists of a hierarchical set of patterns -

The argument pattern which must match in order for the function's computation to even start,

a tree of exception patterns, handlers and restart patterns,

the result value and

side effects if any.

All of them are available as design tools for a function creator so that she can create code that can be used in a variety of contexts not all of which can be known at design time.

An analogy

It is interesting to compare the hierarchy of function dependencies to a company's employee hierarchy. It is often the case that at every designation, an employee has access to some information that employees at lower designations don't. This information lets them make decisions in situations where those working under them don't know enough to make the decision themselves.

Imagine a company P that works like this - a task is handed down to the top-level employee who further describes it and hands subtasks down to those working under her. The company philosophy is that whenever an employee reaches a point at which she can't take a decision, she promptly destroys all the work she's done starting from the time her task was handed to her. Then she informs her boss that such a situation has happened. Her boss is very happy with her capacity to cleanup her work without leaving a trace, destroys all the work she has done since her task was handed down, files a report in the company activity database and talks to her boss about what happened, in her own language.

I'm sure you don't work in a company like that! Now bump up the scale and imagine if Boeing worked like that!

What usually happens is that the lower level employee suspends her work and immediately talks to her boss, presenting her with a few options. Her boss, based on what she understands, instructs her to proceed in a particular direction. The result is that the task gets done. All the way up the hierarchy.

We'll not want to work in the pathological company P, but we all choose to live with and even praise programming languages that work exactly like that.

Appendix

(*) In muSE, it is slightly more efficient to write the G and H functions as -


(define H (fn (x y)
             (try (/ (+ x y)
                     (if (= x y)
                         (raise 'DivideByZeroDiff x y)
                         (- x y)))
               {fn: ('ForceResult r) r})))


(define G (fn (x1 y1 x2 y2)
             (try (/ (H x1 y1) (H x2 y2))
               {fn: (ex 'DivideByZeroDiff x y)
                  (if (= (- x1 y1) (- x2 y2))
                      (ex 1)
                      (retry 'ForceResult 2000))})))

An Objective Scheme

2007-03-13T02:01:00.000+08:00

Here's a first cut at an Objective-C bridge for muSE on MacOSX. There cannot be an easier way to run native code from muSE!

Code is on the objc branch.

Here's a prebuilt universal binary for MacOSX.

The wiki page - AnObjectiveScheme describes it a bit.

Performance

2007-02-14T11:26:00.000+08:00

When you know your code is correct and you don't get any of the diagnostics boxes popping up under Windows or stderr dumps on unices, you may be able to get up to twice the performance of your existing code by turning off the internal diagnostic checks.

To turn of diagnostics, edit the src/muse_config.h file, set the MUSE_DIAGNOSTICS_LEVEL to 0 and recompile muSE.

If performance with diagnostics on is acceptable, you should consider leaving the checks in place.

Subversion perversion

2007-02-02T07:09:00.000+08:00

As I'd mentioned earlier, Google code has added a Wiki feature supposed to be useful to maintain documentation. We in the programming world know that once you document stuff in a place other than the original source code, it rapidly goes out of sync. So I tried another way to present the generated muSE documentation right where the source code is available - the Subversion repo itself.

I simply generated the HTML documentation using doxygen and placed the output into its own Subversion module/folder - here. Once all the files are tagged with the appropriate mime types, the whole site is functional .. and versioned to boot!

Here are the mime-type setting commands you need to use in the doxygen generated documentation folder -


svn propset svn:mime-type "text/html" *.html
svn propset svn:mime-type "image/png" *.png
svn propset svn:mime-type "text/css" *.css
svn propset svn:mime-type "image/gif" *.gif
svn propset svn:mime-type "text/plain" *.md5
svn propset svn:mime-type "text/plain" *.map

A simple module system using macros

2007-01-19T15:45:00.000+08:00

In this post, I describe how to build a simple module system. For simplicity, I assume we won't be providing support for mutually recursive functions within a module.

You can think of a module as a set of symbols and the values they must be bound to. These bindings are made current whenever you "enter" a module's scope and they vanish whenever you exit the scope. Here's what we hope to achieve, with a statistics module as an example -


(define Statistics
  (module 
    (mean 
      (fn args (/ (apply + args) (length args))))

    (variance 
      (fn args (- (apply mean (map (fn (x) (* x x)) args)) 
                  (let ((mx (apply mean args)))
                     (* mx mx)))))
  ))

The above module provides definitions for mean and variance.

Notice that the body of the module definition is very similar to the bindings section of the let construct. We can therefore use the let construct in a macro system to implement module as a macro.

It is simple to use the Statistics module like this -

{Statistics (mean 1 2 3)}

. For that to be possible, Statistics must itself be a macro. Therefore, module must be a macro that generates a macro. Here is a definition for module -


(define module
  (fn 'bindings
    (let ((evaluated-bindings 
            (eval (list 'let bindings 
                    (list 'map 
                          (fn: ((s . _)) 
                            (list s (cons 'quote (eval s)))) 
                          (cons 'quote bindings))))))
      (fn 'args
        (cons 'let (cons evaluated-bindings args))))))

Now all you have to do to rope in the Statistics definitions in a block of code is to wrap it up in a {Statistics ...} block.

Suppose we wish to use the module name to resolve the value of a single symbol. It is potentially expensive to do ({Statistics mean} 1 2 3) because the brace expression places a (potentially huge) let block there which gets evaluated every time the whole expression is evaluated. We can make this case more efficient by defining a scope resolution operator like this -


(define :: (fn 'args (cons quote (eval (eval args)))))

Then we can resolve the meaning of mean like this

({:: Statistics mean} 1 2 3)

. The resolution will happen once at read time, after which the actual function will be used.

See examples/modules.scm.

"Processes" branch is now the "trunk".

2007-01-18T12:57:00.000+08:00

The experimental multi-processing support that was being developed in the processes branch is now merged on to the trunk (v140). That is, its official! I've been waiting for this for a long time now. Apart from various bug fixes and other process related stuff already described, here's whats new -

New primitives post and process?.
New s-expr syntax and functions to work with raw byte data.
MAJOR code change to make muSE truly embeddable. This change breaks API compatibility. The new API calls all take the environment pointer as their first arguments so that multiple muSE environments can coexist in the same OS process or thread without interfering with each other.

Specifying function arguments using key-value pairs

2007-01-10T10:27:00.000+08:00

Since v129 in the processes branch, two new primitives call/keywords and apply/keywords have been added to let function argument be specified in an order that's different from the declaration order. These primitives do not change the evaluation complexity of functions at all, but simply expose a function's declared arguments as "keywords".

(call/keywords f 'key1 val1 'key2 val2 ...): Evaluates the given function by explicitly binding each named argument to the given value. Can also be used on macros, in which case the value positions are used without being evaluated. This is the counterpart of the normal position based evaluation.
(apply/keywords f alist): If you store a set of argument bindings in an alist, you can apply a function to those bindings using apply/keywords. This is the counterpart of the position-based apply.

Any arguments that don't feature in the supplied key-value pairs take on their current value in the environment. This lets you specify common arguments using just a let binding or a global define. Such common arguments are therefore said to be implemented via a limited form of dynamic scoping.

Creating stand-alone executables

2006-12-27T11:37:00.000+08:00

The muSE interpreter binary can now create extensible stand-alone command line executables. For details, check out the wiki page StandAloneExecutables.

Simple run-time type checking in muSE

2006-12-25T08:49:00.000+08:00

The guarded patterns facility can be used to implement a simple type checking facility that can be turned on and off with a global setting. Some functions useful in this situation are defined in examples/rtts1.scm.

muSE has some built-in functions to check types - int?, float?, cons?, vector?, hashtable?, text? and symbol?. The common characteristic of these predicates is to evaluate to their argument if it satisfies the predicate and to () if it doesn't.

If you take a type to be defined by a predicate - the class of objects being all objects that satisfy the predicate - the above predicates together with the combinators ?or, ?and, ?not and ?list-of defined in examples/rtts1.scm can express a broad set of types.

Note: The type checks are all performed at function invocation time. This can be quite a drag on performance. So the definition of decltype in the rtts1.scm file is such that the checks can be turned off by setting the *enable-rtts* to () at the start of the file.

Here's an example (note the use of braces {} around the decltype expressions) -


(define number? (?or int? float?)) ; From rtts1.scm

(define max
  (fn ({decltype n1 number?}
       {decltype n2 number?})
      (if (> n1 n2) n2 n1)))

(define multi-max
  (fn ({decltype n1 number?} . {decltype ns (?list-of number?)})
      (reduce max n1 ns)))

With type checking enabled, multi-max will only be usable with numeric arguments and you have to give it at least one argument.

If you disable type checking, then the above definitions become as though you'd typed -


(define max
  (fn (n1 n2)
      (if (> n1 n2) n2 n1)))

(define multi-max
  (fn (n1 . ns)
      (reduce max n1 ns)))

Google code adds Wiki ...

2006-12-17T16:35:00.000+08:00

I've been blogging about aspects of muSE on this site, but am feeling uncomfortable with the organization that's turning out. When I visited the google code pages a few moments ago, I saw that a new Wiki tab has been added. That's the right tool to put in the kind of documentation I've been placing online using the blog and I'll be exploring moving the documentation that needs to be organized over to the wiki and leaving all the "latest scoop", "how to" and "things to do" kind of posts to the blog.

I could've created a separate web site ... ok just call me a lazy one.

Continuations and processes

2006-12-14T00:27:00.000+08:00

The call/cc implementation has been revamped to make continuations work only within the processes in which they were captured. This is because none of the alternatives to supporting inter-process continuation invocation seemed clean enough or necessary.

What should happen when a process invokes a continuation that was captured by another process? Should it terminate the current process and join with the process to which the continuation belongs? If that's the case, what about the continuations that were captured in the invoking process? Should they be rendered invalid? Would this whole spaghetti be useful or meaningful?

An alternative is to have the continuation invocation finish evaluation in the invoking process and return with a value like nil or T. Doing that, however, means different behaviour when invoking a continuation in the same process it was captured in versus invoking a continuation captured in a different process. In the former case the invocation never completes evaluation whereas in the latter case it does.

Due of all of that, I've disabled invocation of continuations across process boundaries. Any possible use for that can be satisfied by the message passing mechanism (I think), which is simpler and more comprehensible anyway.

Erlang style processes in muSE

2006-12-13T15:36:00.000+08:00

muSE now has an implementation of co-operative message passing processes in the same spirit as Erlang. The source code canbe obtained from the processes branch.

muSE processes provide an abstraction that let you think of your program as concurrently acting entities without worrying about the actual order in which the operations are actually being performed by the processor. Evaluation of muSE expressions may be pre-empted at graph reduction boundaries to pay some attention to other processes. Switching between processes is fairly efficient (close to setjmp + longjmp in C) and it is even possible to run 10000 processes without bringing down your machine to its knees, granted that each process will run pretty slowly on the average in that case though.

Spawning processes

(spawn thunk [attention])

Spawns off the given thunk (a function that takes no parameters) into a separate process. The thunk is evaluated in a loop, until it returns a non-nil value. Once the thunk returns with a non-nil value, the process dies. This is thunk's way of saying "I'm done." The (optional) second argument to spawn is an attention value that tells the muSE scheduler how much attention it should give to the created process before switching to another. The default value is 10. Play around until you find something that suits you. The processes are all scheduled in a simple round-robin manner for now.

The spawn expression itself evaluates to the process-id of the created process. The process id is not a number as is usual in most systems, but is actually a native-closure. The only two uses for the pid are to compare two pids for equality and to pass messages to the process it identifies.

To pass a message, simply use the process ID like a normal function. Its entire argument list will be placed as a single message in the process's message queue.

Pausing a process

(run [timeout-microseconds])

Pauses a process for the given timeout period, yielding time to other processes. If the timeout is omitted, the process is suspended for ever.

Receiving messages in a process

(receive ...)

Retrieves the next message in the process's mailbox. The message has the format

(pid . values)

where pid is the id of the process that sent the message and values is the list of arguments that it supplied to the message sending operation.

The result of a (receive...) expression is designed to be used with muSE's pattern-matching case construct as follows -


(case (receive)
   ((pid ...msg-pattern-1...) action-1)
   ((pid ...msg-pattern-2...) action-2)
   ...)

The receive primitive has four forms -

(receive): Pauses the process until a message is available in the process's mailbox and evaluates to the message. This will always evaluate to a valid message.
(receive timeout-microseconds): Pauses the process for at most timeout-microseconds. If there is mo message in the process's mailbox for more than timeout-microseconds, it evaluates to () which is muSE's false value.
(receive pid): Waits for and retrieves the next message from the process with the given pid.
(receive pid timeout-microseconds): Similar to the previous one, but times out with a () value after timeout-microseconds.

Controlling concurrency

(atomic ...expressions...)

From the point of view of a single process, atomic works exactly like do - evaluating all the expression in turn and itself evaluating to the value of the last expression in the series. From the point of view of the cluster of running processes, it does not yield any time to any other process until all its expressions are evaluated.

atomic expressions may be nested to arbitrary depths. You can forcibly yield to other processes however, if you use any of the pausing functions receive and run.

Process identity

(this-process)

Evaluates to the pid of the process in which it is evaluated.

Embedding issues

For muSE, it is important to consider how these processes interact when embedded in a C/C++ based application. The application can make calls to the muSE API to evaluate expressions. These calls are always evaluated in the "main process" using the main application's C stack. Each muSE process has its own C stack and won't interfere with the main C stack. The entry and exit processes for this case will always be the main process. Other processes get a chance to execute only when within a muSE API call and therefore will not interfere with the application's execution when not running muSE code.

Exception mechanism

2006-12-13T07:44:00.000+08:00

muSE had so far lacked the facility to raise exception conditions and handle them in code that's specified non-locally. One could conceivably use the call/cc construct to implement raising exceptions, but capturing a continuation to evaluate an expression that often won't invoke the continuation turns out to be expensive in terms of memory - capturing a continuation copies the stack in muSE.
As of version 73 in the processes branch, a simple exception raising and handling mechanism has been added to muSE. There are two new primitives involved -

(raise ...args...)

The raise primitive is used to flag an exceptional condition and results in all the established handlers being tried one by one until one of them can be found to handle the condition. A handler (which is a function) is taken to accept an exception for handling if its argument pattern matches the pattern of arguments to the raise expression that raised the exception.

(try expr handler1 handler2...)

The try block wraps the expr with handlers that get tried when any sub-expression of expr raises an exception.

Any muSE object can be used in the place of a handler. If the object is a function, then its arguments have to pattern match against the exception raised in order for its body to be evaluated as the result of its try block. If the object is not a function, its value is used as the result of the try expression as is. For example -


(try (if (< a b) 
         (- b a)
         (raise 'NotInOrder a b))
   0)

will evaluate too 0 if a >= b.

A function used as a handler needs to have its arguments in a special order -

The first argument to the handler is an exception object.

The remaining arguments are the same list of values passed to the raise expression that raised the exception.

For example -

(fn (ex 'NotInOrder x y) ...)

can be the signature of a handler that handles the 'NotInOrder exception raised by the previous example. The ex argument's sole purpose is to let you resume the computation from the raise expression with a new valid value as the result of the raise. The ex object is actually a (cheaper than call/cc) continuation that you can invoke with a single argument that will resume the computation in such a manner. To expand on the previous example,


(try 
  (do (write "Difference = "
             (if (< a b)
                 (- b a)
                 (raise 'NotInOrder a b)))
      (write "Product = " (* a b)))
      
  (fn (ex 'NotInOrder x y)
      (ex (- x y))))

In the above form, the exception condition is corrected by reversing the arguments of the subtraction operation. If the handler did not invoke the exception object to resume the computation, its result value is used as the result value of the try block. Trivial example, yes, but serves to illustrate the point.

Processes branch

2006-12-12T10:36:00.000+08:00

The processes branch of muSE has now reached a fair degree of completeness. This post documents the facilities available as of version 82, whose snapshot is available in the tag v0.2cp. I intend to make the processes branch the main trunk when the functionality is a bit more tested.

The processes branch implements a few significant features over and above the standard muSE engine on the trunk -

Processes: A simple implementation of Erlang style message passing processes in order to express concurrent computations. Primitives added are spawn, receive, atomic and run.
Process-local continuations: Continuation support modified to work within a process. Continuation invocation across process boundaries is forbidden. No new primitives.
Process-aware networking: muSE had some simple s-expr based communication functions all along. This branch now has these functions process aware - i.e. reading from a network port will not stall processes. A polling mechanism (which is invisble at the scheme-level) is used to handle all connections.
Resumable exception mechanism: New primitives try and raise provide the means to raise, handle and resume exceptional conditions. This is something muSE had been lacking all along and was badly needed.
Static guarded patterns (minor)

Static guarded patterns

2006-11-26T17:26:00.000+08:00

muSE makes use of pattern matching binding in its let, case and fn constructs. These patterns can destructure lists and test for equality against numbers, symbols and any quoted constants. In several situations, it is desirable to make the binding operation succeed only if the pattern satisfies a more complex condition that can be expressed only in code.

With revision 36 of muSE trunk, support for static guarded patterns have been added, which allows you to specify conditions that a pattern must satisfy in order for the binding operation to succeed.

In any language with pattern matching binding, a guard typically has two components - 1) a pattern to match and bind and 2) a condition that the bound variables must satisfy additionally, for the binding operation to be considered to be successful. A fairly straight forward s-expression form of a guard therefore is

(guard PATTERN TEST-BODY)

where the PATTERN introduces variables and the TEST-BODY makes use of the introduced variables in an expression that evaluates to a boolean value (in muSE, () is 'false' and anything else is 'true').

We can exploit the fact that the above guard form is structurally and semantically identical to a predicate expressed as a closure and reuse that mechanism to add support for guards in patterns. There is, therefore, no need to introduce another special symbol guard.

Hence, you can place guards wherever a pattern is expected - in let, arguments to fn itself and, most importantly, the case expression. The current implementation of guards does not allow the guard body to refer to the lexical context. It can only refer to the global context and that's why the guard mechanism is called static.

If you use an fn expression directly in a pattern, like

(a b (fn (c d) (< c d)))

against a value '(1 2 (3 4)), it is not possible to tell whether you intended the third element of the list to be decomposed into 3 elements, binding the first to the symbol fn or you intended for the fn expression to be used as a guard. To resolve this ambiguity, we call upon muSE's read-time expression evaluation to really place a predicate object at the third position - like this -

(a b {fn (c d) (< c d)})

. Now, the pattern matcher actually sees a function object in the third position and knows to treat it as a guarded pattern. This pattern will bind 4 variables a, b, c and d if it succeeds.

There is limited support for dynamic guards in patterns. You can use muSE's dynamic scoping mechanism fn: instead of fn in cases where you need the guard body to refer to variables in the immediately enclosing lexical closure.

Multiple muSE instances

2006-11-08T06:43:00.000+08:00

When embedding muSE into an existing C/C++ program, there are a few ground rules for creating and using muSE environments a.k.a. execution contexts.

You are allowed to create multiple muSE environments in a single process.

You have to limit all expression evaluation to a single thread of execution.

You can switch between different environments using the muse_set_current_env() API call. If you don't the last created environment will be the current environment for evaluating expressions.

If you statically link muSE into a shared module (.dll/.so), each shared module gets its own current environment state and will not interfere with other shared libraries.

In the future, the current environment state may become thread-local, in which case you'll be able to use different environments simultaneously in different threads. Still, a single environment will be allowed to be the current environment of only one thread at a time. Note that the API won't need to change to support this behaviour extension.

Polymorphic higher order functions

2006-10-27T09:32:00.000+08:00

Its been a while since my last posting. That's because a lot of changes have been happening under the hood. Objects (vectors and hashtables) now have printed representations, higher order functions are now polymorphic - they can be used on lists, vectors and hashtables, the diagnostic messages are now more extensive and several hashtable bugs have been fixed. There is a fairly serious language departure from standard Scheme which doesn't have polymorphic operators, but I chose this for ease of use. Here, I describe what's available as of revision 21 in the svn repository.

The following operations are now applicable to lists, vectors and hashtables. Wherever you see collection, it means you can use any of those. -

(map (fn (x) ...) collection)

Creates a structurally similar collection with all the values transformed using the given function. The hashtable keys are not processed.

(join list1 list2 ...) (join vector1 vector2 ...) (join hashtable1 hashtable2 ...)

Concatenates all the objects into a single collection. All objects must be of the same type. Lists get concatenated into a single list, vectors get concatenated into a single vector. For hashtables, a union hashtable is created with all the key-value pairs of all the hashtables.

If two hashtables have the same key, the value for that key will be the value in the hashtable that's later in the argument sequence. You can influence this by supplying an optional reduction function as the first argument to join. The default behaviour is as though the reduction function is (fn (v1 v2) v2).

(length collection)

You can use the same length function to get the size of any data structure. A synonym size is available as well for clarity.

(find item collection)

Finds the item in the given collection and returns something that you can use to locate the item within the collection, or () if the item cannot be found. In the case of lists, it returns the remainder of the list starting from the object. In the case of vectors, it returns the index of the item. In the case of hashtables, it returns the key for which the given item is the value.

(reduce (fn (acc x) ...) initial collection)

Uses the given reduction function and reduces the given collection of values to a single value. This is the foldl operation that must be familiar to functional programming afficionados. This function only uses the values in the data structures. The keys of a hashtable, for example, are not touched.

(for-each func collection)

Invokes the function for each entry in the data structure. The result value is always (). The function is expected to have the signature -

(fn (obj) ...)

It is to be noted that the iteration always happens over the value objects. You don't have access to the keys of hashtables or the indices of vectors when using for-each. For a more generalized iteration construct, see collect below. This (new) behaviour is since v120.

(andmap predicate collection)

Returns T if all elements satisfy the predicate and () if even one doesn't. The signature of the predicate has the same signature constraints as that for for-each.

(ormap predicate collection)

Returns () if none of the elements satisfy the predicate. If even one satisfies the predicate, returns a reference into the data structure (like find) using which you can locate the element. The predicate signature constraints are the same as for andmap and for-each. You can use ormap as a generalized version of find.

(collect collection predicate mapper [reduction-fn])

This is an experimental generalized construct that fuses filtering, mapping and reduction operations into a single loop. The predicate selects elements from the collection, the mapper transforms the elements and the (optional) reduction-fn combines multiple values into a single entry in the result.

The predicate has to have the signature (fn (x) ...) for lists. For vectors, it is expected to have the signature (fn (index . value) ...) and for hashtables, it must be (fn (key . value) ...). If you want to run the mapper on every element, you can simply pass () as the predicate - indicating that you don't wish to do the test.

Once an element passes the predicate test, it is passed to the mapper function. If no mapper function is given, the identity map is assumed and in this case collect behaves like a pure filter.

For lists, the mapper function should have the usual signature (fn (x) ...).

For vectors, the signature is expected to be (fn (index . value) ...). The index will be the filtered index of the value and not the original index in the vector. The mapper function is expected to return a pair of the form (result-index . result-value). The result is used to determine where in the result vector the result-value should be placed. The result vector is automatically resized to fit the set of returned indices. The (optional) reduction-fn is used to determine how to combine multiple values if more than one value is mapped to the same result index.

For hashtables, the case is similar to that of vectors, except that instead of index-value pairs, the mapper works with (key . value) pairs and is expected to return a (result-key . result-value) pair. The reduction function is also applied similar to the case of vectors.

Object I/O

2006-09-19T22:10:00.000+08:00

... or Why you don't need to invent a new syntax for every object type in muSE.

Standard Scheme (RnRS) has special read/write syntax for vectors which goes like this - #(1 2 3 4), which will print out as #4(1 2 3 4). muSE doesn't have such special syntax for objects because its syntax for read-time evaluation is general enough to cover vectors, hashtables and (as far as I can see) all new object types that can be added to muSE in the future.

The idea is dead simple and is based on the read-time evaluation done by muSE. It is quite likely that you'll have written a function that takes a list of arguments and constructs your object. In the case of vectors, muSE has vector that's used like (vector 1 2 3 4) and in the case of hashtables, it has hashtable that's used like (hashtable '((key1 . value1) (key2 . value2))).

Whenever the writer encounters an object, it simply has to write it out in that "constructor" function notation, using {} instead of (). When such an expression is read back in by muSE, the reader will expand the braces and return the constructed object directly. For example -
> (define v (vector 1 2 3 4))
> (write v)
{vector 1 2 3 4}

When the reader reads the following expression -
(third item {vector 1 2 3 4} is a vector)
you'll actually get a vector as the third item in the above list.

Anonymous symbols

2006-09-15T03:57:00.001+08:00

muSE has a notion of anonymous symbols - symbols which do not have a textual representation - which are useful in situations where you need to keep a set of properties together, and in macros to introduce new variables into the generated expressions. The difference between named and anonymous symbols (apart from the textual representation) is that named symbols persist for the life time of the muSE execution environment whereas anonymous symbols and their property lists are garbage collected when there are no references to them.

Anonymous symbols are created using (new) . You can use an anonymous symbol as the first argument to the get and put functions to edit its property list.

Property lists

2006-09-15T03:49:00.001+08:00

Every symbol in muSE has an associated property list that you can query and edit using the get and put functions. For example -

> (put 'kumar 'sister 'hamsa)
(sister . hamsa)
> (get 'kumar 'brother)
()
> (get 'kumar 'sister)
(sister . hamsa)

A symbol's property list is globally available and is not changed by local contexts such as let.

Symbols and values

2006-09-15T03:37:00.001+08:00

In muSE, as in all Schemes I guess, named symbols are entities that are uniquely specified by their textual representation - i.e. two symbols with the same name refer to the same internal object, irrespective of context. In muSE, all named symbols are "interned" forever - i.e. they are automatically kept alive for the life of the running environment.

You can bind values to symbols either using the define syntax or using the set! function. There is very little difference between define and set!. They can both assign values to symbols at the lexically top-level, but only define can be used to specify recursive functions. muSE's define syntax is more restrictive than R5RS Scheme in that you can define functions only like this -


(define f (fn (...args..) ...body...))

whereas in standard Scheme you'd define it like this -


(define (f ...args...) ...body...)

It is not an error to define the value of a symbol more than once using define, but it will complain because it is a common source of programming error that indicates that an incorrect assumption is probably being made. set! will not complain, of course, as the intention is clear.

You can get the string name of a symbol using (name sym) and you can intern a symbol given its string representation using (symbol "name").

Differences with standard Scheme
In MzScheme (for example), symbols introduced by define are not closed over when creating functions using lambda. Changing the definition of such a top-level symbol will change the behaviour of the function created using lambda. In muSE, however, fn captures the values of symbols at the time it is being created, including all top-level definitions. fn:, on the other hand, allows its behaviour to be changed after its definition ... even in a local context such as that introduced by let.

Hashtables are functions

2006-09-12T13:52:00.000+08:00

In the same spirit as vectors, hashtables can be thought of as functions that map keys to values. muSE hashtables are presented exactly like that and don't need special accessor functions.


> (define rgb (mk-hashtable))
> (rgb 'red 255)
> (rgb 'green 255)
> (rgb 'blue 255)

If you load the above definitions, you can get an alist from the hash table using -
> (hashtable->alist rgb)
which will give
((red . 255) (green . 255) (blue . 255))
(The order is unspecified, though.)

You can retrieve the green component using (rgb 'green). If you supply a key that is not present in the hashtable, the function will return (). In muSE therefore, it is not possible to distinguish between a key with a value that is () and a key that is not present in the hashtable. This fact is used to remove a key from the hashtable if you pass () as the value argument. For example -


> (rgb 'green ())
> (hashtable->alist rgb)
((red . 255) (blue . 255))

This is not really a restriction and you can use a hashtable as a set by setting the value to any non-NIL value.

Hashtables accept integers, strings and symbols as keys. Therefore, you can use a hashtable like a sparse vector if need be, since they both have the same invocation interface.

Vectors are functions

2006-09-12T13:40:00.000+08:00

A vector is, conceptually, a function from an index to an object and in muSE, a vector is exactly that - a normal function. Here's an example -

(define rgb (mk-vector 3))

Now rgb is a 3-element vector, with all the slots set to (). Here's how to set the three color components -


(rgb 0 255) ; Red
(rgb 1 255) ; Green
(rgb 2 255) ; Blue

To get the green component, for example, you use (rgb 1). If you pass an index out of range, you always get ().

You can create a vector from data using the vector function like this -
(vector 255 255 255)

Other vector manipulation functions are more or less the same as standard Scheme - such as vector-length, list->vector and vector->list.

"Backquote" syntax

2006-09-12T10:26:00.000+08:00

muSE does not support the Scheme/Lisp backquote notation in its reader. This is primarily because I was lazy, but later on I realized that it is simple to implement something like it using muSE's macro facility. Here's the definition -


(define literal 
  (fn 'args
    (case args
      (() 
       ())
      ((('unlit expr) . etc) 
       (list 'cons expr (apply literal etc)))
      ((('unlit-splice expr) . etc) 
       (list 'append! expr (apply literal etc)))
      ((x . etc) 
       (list 'cons (cons quote x) (apply literal etc))))))

For example -


(literal 1 2 (+ 1 2)
          (unlit (+ 2 2))
          (unlit-splice (map (fn (x) (* x x)) '(5 6 7)))
          8 9 10)

will get you the literal expression -
(1 2 (+ 1 2) 4 25 36 49 8 9 10)

This works mostly well enough to write macros using it, except when you want to use a macro-like expression within the literal, in which case the result of the macro expansion will be used instead of the literal macro term. This is due to the tail-first expansion performed by muSE.

Read-time evaluation & first class macros

2006-09-12T07:01:00.000+08:00

muSE has a simple approach to evaluate certain expression at read-time - you enclose the expression in braces {} instead of parentheses (). If you have untrusted input sources, the muSE API lets you turn off read-time evaluation. You can use read-time evaluation to precompute subexpressions that won't change during execution.

Apart from the braces approach, muSE provides a way to specify functions which take in their syntactic arguments - i.e. their unevaluated arguments and can return code in the form of another expression that is evaluated instead. These are called macros - just as in Scheme. A macro is specified in muSE using the fn expression that's used for normal functions, but the entire argument list should be quoted. Here's an example macro that evaluates a three-term infix expression -


(define infix3
 (fn '(x op y)
   (list op x y)))

Macro calls may be enclosed in braces or parentheses - both are accepted. So the following uses infix3 in the expected way -


> (infix3 2 + 3)
5

Macro symbols are recognized at the head of a parenthesized list, but not anywhere else. So you can get the expression that the infix3 macro computes by using apply.


> (apply infix3 '(2 + 3))
(+ 2 3)

This is possible because macros in muSE are first class entities - i.e. they can be passed around by value.

Evaluation order
In common-lisp, I believe macros are expanded head first and they continue to expand until no more macros exist in the expression. muSE, on the contrary, performs tail-first expansion.

For example, in the expression (infix3 2 + (infix3 1 + 2)),
the inner (infix3 1 + 2) is expanded before passing on to the outer infix3, so the outer infix3 sees the expression (2 + (+ 1 2)), which it'll transform to (+ 2 (+ 1 2)).

Braces vs. parentheses
Braces are evaluated even if they occur within quoted expressions, whereas parentheses aren't, even if they contain sub-expressions that look like macro calls. So the expression
'(1 2 {+ 3 4} 5 6)
is actually
'(1 2 7 5 6)

Pattern matching bind

2006-09-12T06:35:00.000+08:00

muSE uniformly uses pattern matching to bind symbols to values. It is used in fn, case and let expressions - which therefore differ slightly from standard Scheme. Not only was it easier to use the same technique in all three expressions, it has resulted in greater expressive power for let and case, obviating the need to do first, rest and such destructuring operations on lists.

muSE's pattern matching bind can deconstruct lists and match constants such as numbers, strings and symbols. Here's an example using fn and case - Suppose we need to create a function that adds up the pair-wise product of its arguments. i.e -

> (f 1 2 3 4 5 6)
should yield -
1 * 2 + 3 * 4 + 5 * 6
= 45

We can write f like this -


(define f
  (fn args
    (case args
      (() 0)
      ((x y . etc) (+ (* x y) (apply f etc))))))

Note that args is used by itself without an enclosing parentheses to get the arguments of the function as a list - this itself is a pattern match. Also note that NIL can be notated as () without a quote character.

Similarly, let also allows you to deconstruct lists. Apart from that, the behaviour of let in muSE is similar to let* in Scheme. There are no other kinds of let in muSE because so far this one has been sufficient.

Functions/closures

2006-09-12T06:13:00.000+08:00

muSE doesn't use the Scheme standard lambda keyword to create functions. This language deecision is because muSE is used by non-programmers who are slightly familiar with JavaScript, but will freak out if they see things like lambda occuring anywhere. It has fn and fn: instead. fn behaves like you'd expect lambda to - capturing the lexical context in a closure. fn: creates a function which has a dynamically scoped body. Here's an example that tells you the difference between the two -


(define y 2.0)
(define f (fn (x) (+ x y)))
(define g (fn: (x) (+ x y)))

Now,


> (f 5.0)
7.0
> (g 5.0)
7.0

Fair enough, but now lets change the definition of y ..... locally!


(let ((y 4.0))
 (print (f 5.0))
 (print (g 5.0)))

Now f continues to use the old value of y whereas g uses the new value of y instead. The above expression will print -


7.0
9.0

API documentation

2006-09-11T19:13:00.000+08:00

muSE API documentation in HTML format now available.

Windows CHM help file

Online HTML docs

The API documentation was generated by running the doxygen tool on the muSE source code.

Welcome to muSE

2006-09-10T19:53:00.000+08:00

Hello and welcome to muSE.

I wrote this Scheme dialect with the intention of having a simple small foot print engine that one can use as an expressive embedded extension language. The company I work for has allowed me to publish muSE under very liberal open-source license terms.

Selling points -

Easy to use and reasonably documented C embedding API.
Closed execution environment - instabilities won't leak into your system unlike an environment that uses the Boehm garbage collector.
Fairly small foot print - WIN32 executable is about 80KB including diagnostics.
64-bit ready - muSE integers are always 64-bit and 64-bit pointers are just a recompile away.
Unicode throughout - Exclusively uses 16-bit characters internally, supports (only) UTF-8 for I/O.
Fast enough as a glue language (at least for us). Startup/shutdown times are also quite fast due to the minimal language.
Suitable for creating simple DSLs.
Some novel/experimental features -

Simple notation for read-time code evaluation.

First class reader macros that expand tail-first.
Dynamic or lexical scoping - you choose according to your need.
Diagnostics that make spell-check-like suggestions.
Experimental support for ez2scm syntax.

What muSE is not -

A full R5RS compliant Scheme. We don't need full support because the runtime is expected to be aggresively extended using C/C++ depending on the usage context. Basics are however covered - map, for-each, list manipulations, closures, call/cc, vectors, hashtables, ports, etc. are in there. If you're looking for a full R5RS Scheme, try MzScheme which is supported by the wonderful DrScheme IDE.
A coffee maker that toasts bagels to boot.