Introducing Objective newLISP

newLISP is an awesome language that I use for all of my scripting needs, but one thing that is missing from it is a nice way of doing real object oriented programming.

By default it supports a pseudo-OOP paradigm called FOOP, but FOOP is simply inadequate for doing some of the most rudimentary of OOP tasks, such as allowing objects to hold references to each other.

That is why I’m announcing Objective newLISP: Real Object Oriented Programming for newLISP.

Let’s Dive In

Objective newLISP—ObjNL for short—is modeled after parts of Objective-C and Java. Let’s open up a REPL and begin:

$ newlisp ObjNL.lsp
newLISP v.10.1.6 on OSX IPv4 UTF-8, execute 'newlisp -h' for more info.

> 

Classes

Classes are simply contexts and are defined using the function new-class:

> (new-class 'Foo)
Foo

If we wanted to create a subclass of Foo called Bar we can easily do so:

> (new-class 'Bar Foo)
Bar

We can see that Foo is the superclass of Bar:

> Bar:@super
Foo

And that all classes inherit from ObjNL:

> Foo:@super
ObjNL

Objects

Objects are instantiated from classes using the function instantiate. They are contexts too:

> (setf obj (instantiate Foo))
Foo#1

As we’re subverting newLISP’s ORO memory management model to gain real OOP, we should deallocate it manually when we’re through using it. I will cover the topic of memory management last.

Constructors

Constructors are defined using the default function. Let’s define constructors for Foo and Bar (suppose we entered this into the REPL between a pair of [cmd][/cmd] tags):

(context Foo)
(define (Foo:Foo _bar)
    (setf bar _bar)
    true
)

(context Bar)
(define (Bar:Bar _foo)
    (setf foo _foo)
    true ; don't allow ourselves to be deallocated
)
(context MAIN)

Note the extra true at the end of each constructor. This is important because if the constructor returns nil that tells ObjNL that an error occurred and to therefore deallocate the object immediately. Thus if _bar were nil and we didn’t have that true the object would be deallocated, and we don’t want that.

When we call instantiate with extra arguments they are passed to the constructor:

> (setf obj (instantiate Foo (instantiate Bar)))
Foo#2

We can see that the instance variables were properly set:

> obj:bar
Bar#1
> obj:bar:foo

ERR: symbol expected : "obj:bar:foo"

Huh. We were able to check obj:bar but obj:bar:foo resulted in an error. It seems newLISP treats the entire thing as a symbol if there’s more than one colon, instead of assuming we’re doing multiple context lookups.

Thankfully Objective newLISP has you covered.

Deep Value and Symbol Access

> (. obj bar foo)
nil

The dot macro lets us look up the value of a symbol that we want through several object references. I’ll refer to this as “deep value access”. Sometime we want the symbol instead of the value, for example say for fun we want to create a circular reference between the objects obj and obj:bar. We can do this using the dot-reference macro:

> (.& obj bar foo)
Bar#1:foo
> (set (.& obj bar foo) obj)
Foo#2

The dot-reference macro allows for “deep symbol access”, it returns the context-qualified symbol for an object’s instance variable. Now we can show that our circular reference works:

> (. obj bar foo bar foo)
Foo#2
> (= obj (. obj bar foo bar foo))
true

Interfaces

Most object oriented systems have the concept of an interface, sometimes referred to as a protocol. Interfaces define a set of functions that a class can choose to implement or “conform” to. Objective newLISP has them too, and refers to them as interfaces even though they are technically mixins.

Let’s define a simple interface called protocol:

> (define (protocol:test) "hello!")
(lambda () "hello!")

There are two ways to implement an interface. You can specify a list of them when creating a new class:

> (new-class 'Foo ObjNL '(protocol))
Foo

Or you can add them to a class or object after its definition. We actually want to do this right now because we instantiated obj prior to adding protocol to Foo‘s list of interfaces. We can check to see this is true by asking if obj implements protocol:

> (implements? protocol obj)
nil

So the second way to add an interface to an object or class is to use the function add-interface:

> (add-interface protocol obj)
(protocol Foo ObjNL)

Now obj should implement it, so we can try it out:

> (if (implements? protocol obj) (obj:test))
"hello!"

The only real difference between an interface and a class is that a class has a constructor (default function) and ultimately inherits from ObjNL. You can use implements? to check inheritance as well:

> (implements? ObjNL obj)
true

Memory Management

The last, and perhaps most important topic, is what to do with all those objects you’ve got lying around, also referred to as “memory management.”

Objective newLISP supports two styles of memory management: manual and reference counting.

Manual memory management is simple: instantiate your object, and when you’re done with it, deallocate it!

> (setf b (instantiate Bar))
Bar#2
> (deallocate b)
true

Reference counting is done the same way it is done in Objective-C. Each object starts with a reference count of 1. When you want to hold onto that object you retain it, and when you’re through with it you release or autorelease it (which decrements the reference count). When the reference count hits zero the object is deallocated by deallocate:

> (setf b (instantiate Bar))
Bar#3
> (release b)
true

I will cover autorelease next, but I won’t go to great lengths to explain how all of this reference counting stuff works. If you’re unfamiliar with it, just know that it’s not complicated. If you want some practice make an iPhone app. 😛

To illustrate autorelease I will implement the method ObjNL:dealloc, which is called on an object just before it is deallocated.

> (define (Bar:dealloc) (println Bar:@self " has been deallocated!"))
(lambda () (println "Object " Bar:@self " has been deallocated!"))
> (push-autorelease-pool)
(())
> (dotimes (_ 5) (autorelease (instantiate Bar)))
Bar#8
> (pop-autorelease-pool)
Bar#8 has been deallocated!
Bar#7 has been deallocated!
Bar#6 has been deallocated!
Bar#5 has been deallocated!
Bar#4 has been deallocated!
true

One important point to mention is that deallocating objects in newLISP versions 10.1.8 or older is very slow. The details of why this is has to do with safety (which I discuss in the box below), but needless to say it was too slow to be acceptable. I contacted Lutz Mueller, the author of newLISP, and he agreed to introduce an “unsafe” optimization into the delete function. In versions 10.1.9 and later, deallocating Objective newLISP objects is approximately 480 times faster.

Because of this, it’s strongly recommended to use Objective newLISP with newLISP 10.1.9 or later. Currently the latest development release is 10.1.8, however Lutz graciously made this optimization available online in a development version of 10.1.9. Click here to grab the source for this version. This link will expire soon, when it does you can get the latest development release here.

Cautionary Note!

There are two situations to watch out for when using Objective newLISP:

#1: Unbound References in Functions

Instead of this:

(define (modify-obj)
    (setf obj:bar 5)
)
(setf obj (instantiate Foo))
(modify-obj)

Do this:

(define (modify-obj obj)
    (setf obj:bar 5)
)
(setf obj (instantiate Foo))
(modify-obj obj)

If you don’t do that, newLISP will read the obj:bar in the definition of modify-obj and instantly create and protect a context called obj, making it impossible to setf the obj later on.

#2: Dangling References

Use extreme caution when holding reference(s) to an object in a list or some other container! If that reference is later deallocated and you try to access it, bad things will happen:

> (setf b (instantiate Bar))
Bar#9
> (push b alist)
Bar#9
> (deallocate b)
Bar#9 has been deallocated!
true
> alist
Bus error

Normally this would not be a problem, the object in alist would simply be replaced with nil upon its deallocation. However, since we’re using the fast, unsafe version of delete to do our deallocation, newLISP will not do that. It is the same situation as when attempting to access free’d memory in C/C++/Objective-C.

Instead we should use retain/release:

> (setf b (instantiate Bar))
Bar#9
> (push (retain b) alist)
Bar#9
> (release b)
nil
> alist
(Bar#9)
> (release (pop alist))
Bar#9 has been deallocated!
true
> alist
()

When to use FOOP

Objective newLISP is not the answer to all OOP problems in newLISP. FOOP has its place too. If you’re dealing with a situation where you may end up needing lots of objects, FOOP is probably the better choice. Although you can’t do full-blown OOP with it, FOOP objects can use far less memory than ObjNL objects because in ObjNL, methods are stored in each object, not in the class. After trying out both you should have a good feeling for when to use one over the other (i.e., if the limitations of FOOP start to become obvious).

Download and API

You can grab Objective newLISP from Github.

Access the Objective newLISP API.

And for news, follow @taoeffect and @newlisp on twitter.

Thanks for checking out Objective newLISP!

Enjoy! 😀

5 thoughts on “Introducing Objective newLISP

  1. Reply

    bairui

    This was three years ago now, at version 10.1.6 and newLISP is now up to 10.4.4. Has newLISP made any changes to its FOOP system that render ObjNL obsolete, or is this still a viable alternative to FOOP?

  2. Reply

    Greg Slepak Post author

    I’ve been away from the newLISP scene for several months, so I can’t be sure, but I really really doubt that Lutz introduced any changes to the FOOP system that would address the issue that Objective newLISP addresses, because it would most likely involve a completely rewrite of newLISP’s memory management model.

    1. Reply

      Thomas Grzybowski

      Is there any feedback from users since this posting?

      1. Reply

        Greg Slepak Post author

        Nope! (EDIT: at least not to my knowledge, though I haven’t been paying attention to the newLISP forums.)

  3. Reply

    Mike Manilone

    3 years 😉

    I’m now in great need of a real OO framework in newLISP. This really helps!
    Thanks for your great work!

Leave a Reply to Thomas Grzybowski Cancel reply

Your email address will not be published. Required fields are marked *