Creating stand-alone executables
The muSE interpreter binary can now create extensible stand-alone command line executables. For details, check out the wiki page StandAloneExecutables.
A small footprint embeddable open-source Scheme dialect.
The muSE interpreter binary can now create extensible stand-alone command line executables. For details, check out the wiki page StandAloneExecutables.
The guarded patterns facility can be used to implement a simple type checking facility that can be turned on and off with a global setting. Some functions useful in this situation are defined in examples/rtts1.scm.
muSE has some built-in functions to check types - int?, float?, cons?, vector?, hashtable?, text? and symbol?. The common characteristic of these predicates is to evaluate to their argument if it satisfies the predicate and to () if it doesn't.
If you take a type to be defined by a predicate - the class of objects being all objects that satisfy the predicate - the above predicates together with the combinators ?or, ?and, ?not and ?list-of defined in examples/rtts1.scm can express a broad set of types.
Note: The type checks are all performed at function invocation time. This can be quite a drag on performance. So the definition of decltype in the rtts1.scm file is such that the checks can be turned off by setting the *enable-rtts* to () at the start of the file.
(define number? (?or int? float?)) ; From rtts1.scm
(define max
(fn ({decltype n1 number?}
{decltype n2 number?})
(if (> n1 n2) n2 n1)))
(define multi-max
(fn ({decltype n1 number?} . {decltype ns (?list-of number?)})
(reduce max n1 ns)))
(define max
(fn (n1 n2)
(if (> n1 n2) n2 n1)))
(define multi-max
(fn (n1 . ns)
(reduce max n1 ns)))
I've been blogging about aspects of muSE on this site, but am feeling uncomfortable with the organization that's turning out. When I visited the google code pages a few moments ago, I saw that a new Wiki tab has been added. That's the right tool to put in the kind of documentation I've been placing online using the blog and I'll be exploring moving the documentation that needs to be organized over to the wiki and leaving all the "latest scoop", "how to" and "things to do" kind of posts to the blog.
I could've created a separate web site ... ok just call me a lazy one.
The call/cc implementation has been revamped to make continuations work only within the processes in which they were captured. This is because none of the alternatives to supporting inter-process continuation invocation seemed clean enough or necessary.
What should happen when a process invokes a continuation that was captured by another process? Should it terminate the current process and join with the process to which the continuation belongs? If that's the case, what about the continuations that were captured in the invoking process? Should they be rendered invalid? Would this whole spaghetti be useful or meaningful?
An alternative is to have the continuation invocation finish evaluation in the invoking process and return with a value like nil or T. Doing that, however, means different behaviour when invoking a continuation in the same process it was captured in versus invoking a continuation captured in a different process. In the former case the invocation never completes evaluation whereas in the latter case it does.
Due of all of that, I've disabled invocation of continuations across process boundaries. Any possible use for that can be satisfied by the message passing mechanism (I think), which is simpler and more comprehensible anyway.
Labels: Processes
muSE now has an implementation of co-operative message passing processes in the same spirit as Erlang. The source code canbe obtained from the processes branch.
muSE processes provide an abstraction that let you think of your program as concurrently acting entities without worrying about the actual order in which the operations are actually being performed by the processor. Evaluation of muSE expressions may be pre-empted at graph reduction boundaries to pay some attention to other processes. Switching between processes is fairly efficient (close to setjmp + longjmp in C) and it is even possible to run 10000 processes without bringing down your machine to its knees, granted that each process will run pretty slowly on the average in that case though.
(pid . values)where pid is the id of the process that sent the message and values is the list of arguments that it supplied to the message sending operation.
(case (receive)
((pid ...msg-pattern-1...) action-1)
((pid ...msg-pattern-2...) action-2)
...)Labels: Processes
muSE had so far lacked the facility to raise exception conditions and handle them in code that's specified non-locally. One could conceivably use the call/cc construct to implement raising exceptions, but capturing a continuation to evaluate an expression that often won't invoke the continuation turns out to be expensive in terms of memory - capturing a continuation copies the stack in muSE.
As of version 73 in the processes branch, a simple exception raising and handling mechanism has been added to muSE. There are two new primitives involved -
The raise primitive is used to flag an exceptional condition and results in all the established handlers being tried one by one until one of them can be found to handle the condition. A handler (which is a function) is taken to accept an exception for handling if its argument pattern matches the pattern of arguments to the raise expression that raised the exception.
The try block wraps the expr with handlers that get tried when any sub-expression of expr raises an exception.
Any muSE object can be used in the place of a handler. If the object is a function, then its arguments have to pattern match against the exception raised in order for its body to be evaluated as the result of its try block. If the object is not a function, its value is used as the result of the try expression as is. For example -
(try (if (< a b)
(- b a)
(raise 'NotInOrder a b))
0)will evaluate too 0 if a >= b.A function used as a handler needs to have its arguments in a special order -
(fn (ex 'NotInOrder x y) ...)can be the signature of a handler that handles the 'NotInOrder exception raised by the previous example. The ex argument's sole purpose is to let you resume the computation from the raise expression with a new valid value as the result of the raise. The ex object is actually a (cheaper than call/cc) continuation that you can invoke with a single argument that will resume the computation in such a manner. To expand on the previous example,
(try
(do (write "Difference = "
(if (< a b)
(- b a)
(raise 'NotInOrder a b)))
(write "Product = " (* a b)))
(fn (ex 'NotInOrder x y)
(ex (- x y))))
Labels: Exceptions
The processes branch of muSE has now reached a fair degree of completeness. This post documents the facilities available as of version 82, whose snapshot is available in the tag v0.2cp. I intend to make the processes branch the main trunk when the functionality is a bit more tested.
The processes branch implements a few significant features over and above the standard muSE engine on the trunk -
Labels: Processes
muSE makes use of pattern matching binding in its let, case and fn constructs. These patterns can destructure lists and test for equality against numbers, symbols and any quoted constants. In several situations, it is desirable to make the binding operation succeed only if the pattern satisfies a more complex condition that can be expressed only in code.
With revision 36 of muSE trunk, support for static guarded patterns have been added, which allows you to specify conditions that a pattern must satisfy in order for the binding operation to succeed.
In any language with pattern matching binding, a guard typically has two components - 1) a pattern to match and bind and 2) a condition that the bound variables must satisfy additionally, for the binding operation to be considered to be successful. A fairly straight forward s-expression form of a guard therefore is
(guard PATTERN TEST-BODY)where the PATTERN introduces variables and the TEST-BODY makes use of the introduced variables in an expression that evaluates to a boolean value (in muSE, () is 'false' and anything else is 'true').
(a b (fn (c d) (< c d)))against a value '(1 2 (3 4)), it is not possible to tell whether you intended the third element of the list to be decomposed into 3 elements, binding the first to the symbol fn or you intended for the fn expression to be used as a guard. To resolve this ambiguity, we call upon muSE's read-time expression evaluation to really place a predicate object at the third position - like this -
(a b {fn (c d) (< c d)}). Now, the pattern matcher actually sees a function object in the third position and knows to treat it as a guarded pattern. This pattern will bind 4 variables a, b, c and d if it succeeds.
Labels: Language
When embedding muSE into an existing C/C++ program, there are a few ground rules for creating and using muSE environments a.k.a. execution contexts.
Labels: API
Its been a while since my last posting. That's because a lot of changes have been happening under the hood. Objects (vectors and hashtables) now have printed representations, higher order functions are now polymorphic - they can be used on lists, vectors and hashtables, the diagnostic messages are now more extensive and several hashtable bugs have been fixed. There is a fairly serious language departure from standard Scheme which doesn't have polymorphic operators, but I chose this for ease of use. Here, I describe what's available as of revision 21 in the svn repository.
The following operations are now applicable to lists, vectors and hashtables. Wherever you see collection, it means you can use any of those. -
(fn (obj) ...)It is to be noted that the iteration always happens over the value objects. You don't have access to the keys of hashtables or the indices of vectors when using for-each. For a more generalized iteration construct, see collect below. This (new) behaviour is since v120.
Labels: Data structures
... or Why you don't need to invent a new syntax for every object type in muSE.
Standard Scheme (RnRS) has special read/write syntax for vectors which goes like this - #(1 2 3 4), which will print out as #4(1 2 3 4). muSE doesn't have such special syntax for objects because its syntax for read-time evaluation is general enough to cover vectors, hashtables and (as far as I can see) all new object types that can be added to muSE in the future.
The idea is dead simple and is based on the read-time evaluation done by muSE. It is quite likely that you'll have written a function that takes a list of arguments and constructs your object. In the case of vectors, muSE has vector that's used like (vector 1 2 3 4) and in the case of hashtables, it has hashtable that's used like (hashtable '((key1 . value1) (key2 . value2))).
Whenever the writer encounters an object, it simply has to write it out in that "constructor" function notation, using {} instead of (). When such an expression is read back in by muSE, the reader will expand the braces and return the constructed object directly. For example -
> (define v (vector 1 2 3 4))
> (write v)
{vector 1 2 3 4}
When the reader reads the following expression -
(third item {vector 1 2 3 4} is a vector)
you'll actually get a vector as the third item in the above list.
Labels: macros
muSE has a notion of anonymous symbols - symbols which do not have a textual representation - which are useful in situations where you need to keep a set of properties together, and in macros to introduce new variables into the generated expressions. The difference between named and anonymous symbols (apart from the textual representation) is that named symbols persist for the life time of the muSE execution environment whereas anonymous symbols and their property lists are garbage collected when there are no references to them.
Anonymous symbols are created using (new) . You can use an anonymous symbol as the first argument to the get and put functions to edit its property list.
Labels: Data structures, Symbols
Every symbol in muSE has an associated property list that you can query and edit using the get and put functions. For example -> (put 'kumar 'sister 'hamsa)
(sister . hamsa)
> (get 'kumar 'brother)
()
> (get 'kumar 'sister)
(sister . hamsa)
A symbol's property list is globally available and is not changed by local contexts such as let.
Labels: Symbols
In muSE, as in all Schemes I guess, named symbols are entities that are uniquely specified by their textual representation - i.e. two symbols with the same name refer to the same internal object, irrespective of context. In muSE, all named symbols are "interned" forever - i.e. they are automatically kept alive for the life of the running environment.
You can bind values to symbols either using the define syntax or using the set! function. There is very little difference between define and set!. They can both assign values to symbols at the lexically top-level, but only define can be used to specify recursive functions. muSE's define syntax is more restrictive than R5RS Scheme in that you can define functions only like this -
(define f (fn (...args..) ...body...))
whereas in standard Scheme you'd define it like this -
(define (f ...args...) ...body...)
It is not an error to define the value of a symbol more than once using define, but it will complain because it is a common source of programming error that indicates that an incorrect assumption is probably being made. set! will not complain, of course, as the intention is clear.
You can get the string name of a symbol using (name sym) and you can intern a symbol given its string representation using (symbol "name").
Differences with standard Scheme
In MzScheme (for example), symbols introduced by define are not closed over when creating functions using lambda. Changing the definition of such a top-level symbol will change the behaviour of the function created using lambda. In muSE, however, fn captures the values of symbols at the time it is being created, including all top-level definitions. fn:, on the other hand, allows its behaviour to be changed after its definition ... even in a local context such as that introduced by let.
Labels: Symbols
In the same spirit as vectors, hashtables can be thought of as functions that map keys to values. muSE hashtables are presented exactly like that and don't need special accessor functions.
> (define rgb (mk-hashtable))
> (rgb 'red 255)
> (rgb 'green 255)
> (rgb 'blue 255)
If you load the above definitions, you can get an alist from the hash table using -
> (hashtable->alist rgb)
which will give
((red . 255) (green . 255) (blue . 255))
(The order is unspecified, though.)
You can retrieve the green component using (rgb 'green). If you supply a key that is not present in the hashtable, the function will return (). In muSE therefore, it is not possible to distinguish between a key with a value that is () and a key that is not present in the hashtable. This fact is used to remove a key from the hashtable if you pass () as the value argument. For example -
> (rgb 'green ())
> (hashtable->alist rgb)
((red . 255) (blue . 255))
This is not really a restriction and you can use a hashtable as a set by setting the value to any non-NIL value.
Hashtables accept integers, strings and symbols as keys. Therefore, you can use a hashtable like a sparse vector if need be, since they both have the same invocation interface.
Labels: Data structures
A vector is, conceptually, a function from an index to an object and in muSE, a vector is exactly that - a normal function. Here's an example -
(define rgb (mk-vector 3))
Now rgb is a 3-element vector, with all the slots set to (). Here's how to set the three color components -
(rgb 0 255) ; Red
(rgb 1 255) ; Green
(rgb 2 255) ; Blue
To get the green component, for example, you use (rgb 1). If you pass an index out of range, you always get ().
You can create a vector from data using the vector function like this -
(vector 255 255 255)
Other vector manipulation functions are more or less the same as standard Scheme - such as vector-length, list->vector and vector->list.
Labels: Data structures
muSE does not support the Scheme/Lisp backquote notation in its reader. This is primarily because I was lazy, but later on I realized that it is simple to implement something like it using muSE's macro facility. Here's the definition -
(define literal
(fn 'args
(case args
(()
())
((('unlit expr) . etc)
(list 'cons expr (apply literal etc)))
((('unlit-splice expr) . etc)
(list 'append! expr (apply literal etc)))
((x . etc)
(list 'cons (cons quote x) (apply literal etc))))))
For example -
(literal 1 2 (+ 1 2)
(unlit (+ 2 2))
(unlit-splice (map (fn (x) (* x x)) '(5 6 7)))
8 9 10)
will get you the literal expression -
(1 2 (+ 1 2) 4 25 36 49 8 9 10)
This works mostly well enough to write macros using it, except when you want to use a macro-like expression within the literal, in which case the result of the macro expansion will be used instead of the literal macro term. This is due to the tail-first expansion performed by muSE.
Labels: macros
muSE has a simple approach to evaluate certain expression at read-time - you enclose the expression in braces {} instead of parentheses (). If you have untrusted input sources, the muSE API lets you turn off read-time evaluation. You can use read-time evaluation to precompute subexpressions that won't change during execution.
Apart from the braces approach, muSE provides a way to specify functions which take in their syntactic arguments - i.e. their unevaluated arguments and can return code in the form of another expression that is evaluated instead. These are called macros - just as in Scheme. A macro is specified in muSE using the fn expression that's used for normal functions, but the entire argument list should be quoted. Here's an example macro that evaluates a three-term infix expression -
(define infix3
(fn '(x op y)
(list op x y)))
Macro calls may be enclosed in braces or parentheses - both are accepted. So the following uses infix3 in the expected way -
> (infix3 2 + 3)
5
Macro symbols are recognized at the head of a parenthesized list, but not anywhere else. So you can get the expression that the infix3 macro computes by using apply.
> (apply infix3 '(2 + 3))
(+ 2 3)
This is possible because macros in muSE are first class entities - i.e. they can be passed around by value.
Evaluation order
In common-lisp, I believe macros are expanded head first and they continue to expand until no more macros exist in the expression. muSE, on the contrary, performs tail-first expansion.
For example, in the expression (infix3 2 + (infix3 1 + 2)),
the inner (infix3 1 + 2) is expanded before passing on to the outer infix3, so the outer infix3 sees the expression (2 + (+ 1 2)), which it'll transform to (+ 2 (+ 1 2)).
Braces vs. parentheses
Braces are evaluated even if they occur within quoted expressions, whereas parentheses aren't, even if they contain sub-expressions that look like macro calls. So the expression
'(1 2 {+ 3 4} 5 6)
is actually
'(1 2 7 5 6)
Labels: macros
muSE uniformly uses pattern matching to bind symbols to values. It is used in fn, case and let expressions - which therefore differ slightly from standard Scheme. Not only was it easier to use the same technique in all three expressions, it has resulted in greater expressive power for let and case, obviating the need to do first, rest and such destructuring operations on lists.
muSE's pattern matching bind can deconstruct lists and match constants such as numbers, strings and symbols. Here's an example using fn and case - Suppose we need to create a function that adds up the pair-wise product of its arguments. i.e -
> (f 1 2 3 4 5 6)
should yield -
1 * 2 + 3 * 4 + 5 * 6
= 45
We can write f like this -
(define f
(fn args
(case args
(() 0)
((x y . etc) (+ (* x y) (apply f etc))))))
Note that args is used by itself without an enclosing parentheses to get the arguments of the function as a list - this itself is a pattern match. Also note that NIL can be notated as () without a quote character.
Similarly, let also allows you to deconstruct lists. Apart from that, the behaviour of let in muSE is similar to let* in Scheme. There are no other kinds of let in muSE because so far this one has been sufficient.
Labels: Language
muSE doesn't use the Scheme standard lambda keyword to create functions. This language deecision is because muSE is used by non-programmers who are slightly familiar with JavaScript, but will freak out if they see things like lambda occuring anywhere. It has fn and fn: instead. fn behaves like you'd expect lambda to - capturing the lexical context in a closure. fn: creates a function which has a dynamically scoped body. Here's an example that tells you the difference between the two -
(define y 2.0)
(define f (fn (x) (+ x y)))
(define g (fn: (x) (+ x y)))
Now,
> (f 5.0)
7.0
> (g 5.0)
7.0
Fair enough, but now lets change the definition of y ..... locally!
(let ((y 4.0))
(print (f 5.0))
(print (g 5.0)))
Now f continues to use the old value of y whereas g uses the new value of y instead. The above expression will print -
7.0
9.0