Inside geoWrite – 1: The Overlay System

geoWrite is a WYSIWYG rich text editor for the Commodore 64 GEOS operating system, which runs with a total of just 64 KB of RAM. In the series about the internals of geoWrite, this article discusses how it manages to fit 52 KB of code into the available 23 KB of application RAM.

Introduction

GEOS is a disk-based graphical operating system for the Commodore 64 that provides the following features:

• applications, desk accessories
• disk, printer and mouse drivers
• loadable proportional fonts
• menu bars, dialogs, file picker
• multi-fork filesystem API
• misc. library code (math, memory, strings, …)

But since the OS kernel (called the “GEOS KERNAL”) is only 20 KB in size, some of the APIs are very limited, or cumbersome to use, so applications had to do a lot of work one would expect from the OS these days.

The GEOS authors “Berkeley Softworks” also wrote several applications – the OS-included deskTop, geoWrite and geoPaint, as well as geoPublish, geoCalc, geoChart, geoFile and geoDex – which all share low-level functionality that is not in fact part of the operating system, but shared code between these apps, like:

• code overlays
• custom screen recovery
• create/open/exit startup dialog
• desk accessory enumeration
• font management
• zero page management
• copy protection

In this series of articles, we will discuss some of the lower-level features that are implemented on the application side, using the example of geoWrite.

1. The Overlay System ← this article
2. Screen Recovery
3. Font Management
4. Zero Page
5. Copy Protection
6. Localization
7. File Format and Pagination
8. Copy & Paste
9. Keyboard Handling

GEOS Memory Map

GEOS and its apps need to fit into the 64 KB of RAM of the C64. Here is a rough overview of the memory map, mostly to scale:

-----------------------------------------
$0000  Zero page, stack, system variables
-----------------------------------------
$0400




       Application memory





-----------------------------------------
$6000
       Background bitmap

-----------------------------------------
$8000  System buffers and variables
-----------------------------------------
$9000  Disk driver
-----------------------------------------
$A000
       Screen bitmap

-----------------------------------------
$C000  GEOS KERNAL


-----------------------------------------

The screen bitmap is an 8 KB RAM area for the 320×200 monochrome screen bitmap. There is a full-size “background” copy used for recovering the main screen contents after closing a menu or a dialog without needing a slow redraw.

The application has 23 KB of memory that it can use for code and data. geoWrite is 52 KB of code, and it needs 2 KB for variables, 7 KB for the current page of text and 6 KB for bitmap fonts. That would be 67 KB…

VLIR Files

In the UNIX world, a file is a mapping of a filename to a sequence of bytes (and some metadata). Some operating systems extend this concept: On classic MacOS, files consist of two of these sequences (the “resource fork” and the “data fork”). NextSTEP and MacOS X can present folders with a tree of individual files as a single “bundle”.

GEOS extends the UNIX-like Commodore filesystem with “VLIR” files, which stands for “Variable Length Index Record”. A VLIR file maps a filename to a 256 byte “file header” (for the icon and extra metadata) and up to 127 records, numbered 0-126. A record is a variable-length sequence of bytes, much like a traditional UNIX file.

You can imagine a VLIR file as a folder with several files in it. The following is a visualization of a typical geoWrite document:

geoWrite Doc
\--- File Header
\--- 0
\--- 1
\--- 2
\--- 61
\--- 62
\--- 64

As you can see, record numbers don’t have to be contiguous. (geoWrite for example stores pages in records 0-60, the header and footer into records 61 and 62, and image data in records 64-126.)

The file header contains the icon, the file type, and some other generic as well as application-specific metadata.

VLIR Applications

GEOS applications can be VLIR files as well. When running an app, the system loads record 0 into memory and executes it. It’s entirely up to the application what to do with the other records.

This is what the geoWrite app looks like:

GEOWRITE           size
\--- File Header    256
\--- 0            10335
\--- 1             2552
\--- 2             3999
\--- 3             2328
\--- 4             1965
\--- 5             3870
\--- 6             3998
\--- 7             3897
\--- 8             1194

The geoWrite main code in record 0 is about 10 KB in size. The operating system loads it to $0400-$2C5E into application RAM.

Code Overlays

This is the memory layout of application RAM for geoWrite:

-----------------------------------------
$0400  Main code (record 0)


-----------------------------------------
$2C5F  Variables
-----------------------------------------
$3244  Overlay code (records 1-7)

-----------------------------------------
$41E4  Variables, page data, font data





-----------------------------------------

The record 0 code loaded by the OS always remains in its slot. It contains the core editing functionality, the overlay manager, the font manager and other library code.

There is a 4 KB slot for “overlay” code, meaning that the record 0 code can swap in any of the records from 1 through 7.

• [0 library code, core text editing]
• 1 initialization, copy protection
• 2 core text editing
• 3 cut, copy, paste
• 4 ruler editing
• 5 startup/about, create, open, paste text, run desk accessory
• 6 navigation, search/replace, header/footer, reflow
• 7 printing
• [8 print settings]

(Record 8 is handled differently and is discussed at the end of this article.)

Every record is linked to the same address ($3244) and starts with a jump table, like this one:

CODE5:
    jmp recover            ; 0
    jmp showStartupMenu    ; 1
    jmp renameDocument     ; 2
    jmp openDocument       ; 3
    jmp showAboutDialog    ; 4
    jmp loadDeskAcc        ; 5
    jmp exitToDesktop      ; 6
    jmp readReservedRecord ; 7
    jmp makeFullPageWide   ; 8

The jump table means that the record 0 code can be assembled independently of the overlays. While the overlays access symbols in the record 0 code, record 0 code only calls through these jump table entries.

VLIR API

The GEOS KERNAL has the following calls for working with VLIR files:

• OpenRecordFile – Open an existing VLIR file given its name
• UpdateRecordFile – Flush the VLIR’s metadata to disk
• CloseRecordFile – Flush and close VLIR file
• PointRecord – Set current record
• PreviousRecord – Move to previous record
• NextRecord – Move to next record
• ReadRecord – Read complete record into memory
• WriteRecord – Write/overwrite complete record from memory image
• DeleteRecord – Delete current record

Reading overlay code should therefore be as simple as this:

    ; startup
    LoadW   r0, fnBuffer
    jsr     OpenRecordFile

    ; load overlay code
    lda     #n
loadCode:
    jsr     PointRecord
    LoadW   r7, OVERLAY_ADDRESS
    LoadW   r2, OVERLAY_SIZE
    jsr     ReadRecord

Unfortunately, GEOS can only have one VLIR file open at a time, and a geoWrite document is also a VLIR file. Opening and closing the two files would cause too much disk activity, which is why geoWrite comes with a simple read-only VLIR implementation on the side.

Loading Overlays Manually

On disk, a VLIR file’s directory entry points to it 256 bytes index table. Here is an example:

00 FF  06 13  08 10  09 14  0A 01  0A 12  0A 13  0B 00
0C 02  0D 04  00 00  00 00  00 00  00 00  00 00  00 00
[...]

The 00 FF at the beginning is the Commodore DOS sector header and not part of the data. The remaining pairs of bytes point to the track and sector of the start of each record on disk.

GEOS has an API for loading a file given a track and a sector (ReadFile), so all geoWrite needs to do is read its own index table on startup, and call ReadFile on items of this table when loading an overlay.

Here’s a shortened version of the code to get a copy of the app’s index table:

    ; find application
    LoadW   r6, fnBuffer
    lda     #APPLICATION
    sta     r7L
    lda     #1 ; find max. 1 file
    sta     r7H
    LoadW   r10, appname
    jsr     FindFTypes

    LoadW   r0, fnBuffer
    jsr     OpenRecordFile

    jsr     i_MoveData ; copy index table
    .word   fileHeader+2
    .word   appIndexTable
    .word   2 * NUM_APP_RECORDS
    LoadB   curCodeRecord, $FF
    rts

appname:
    .byte   "geoWrite    V2.1",0

OpenRecordFile reads the VLIR file’s index table info fileHeader. geoWrite then copies NUM_APP_RECORDS into its own table appIndexTable, skipping the first two bytes (00 FF).

And here is a shortened version of the code to read a record:

    ; load overlay code
    lda     #n
loadCode:
    cmp     curCodeRecord
    beq     @rts ; already loaded
    sta     curCodeRecord
    asl     a
    tay
    lda     appIndexTable,y
    sta     r1L
    lda     appIndexTable+1,y
    sta     r1H
    LoadW   r7, OVERLAY_ADDRESS
    LoadW   r2, OVERLAY_SIZE
    jsr     _ReadFile
@rts:
    rts

You can see that the code keeps track of the currently loaded record, so it does not re-load the same code if it’s already in memory.

Managing Overlays

All overlay functionality is implemented in the record 0 code, because it always needs to be accessible.

There is a set of functions for loading the different records:

loadCode1:
    lda     #1
    .byte   $2C ; skip next
loadCode2:
    lda     #2
    .byte   $2C ; skip next
loadCode3:
    lda     #3
    .byte   $2C ; skip next
loadCode4:
    lda     #4
    .byte   $2C ; skip next
loadCode5:
    lda     #5
    .byte   $2C ; skip next
loadCode6:
    lda     #6
    .byte   $2C ; skip next
loadCode7:
    lda     #7
loadCode:
    [...]

The record 0 code can then load an overlay and call a function through its jump table

    jsr     loadCode5
    jsr     J5_showStartupMenu ; OVERLAY_ADDRESS + 3 * 1

Code inside an overlay can’t call code from a different overlay this way, because the loadCode call would overwrite the caller. For this case, the record 0 code has functions like this one:

_showCantAddPages:
    ldy     #<J3_showCantAddPages ; OVERLAY_ADDRESS + 3 * 8
    .byte   $2C
_showTooManyPages:
    ldy     #<J3_showTooManyPages ; OVERLAY_ADDRESS + 3 * 7
    .byte   $2C
_splitTooBigPage:
    ldy     #<J3_splitTooBigPage  ; OVERLAY_ADDRESS + 3 * 5
    ldx     #BANK_3
callRestore:
    lda     curCodeRecord
    pha
    sty     @1
    txa
    jsr     loadCode
@1 = * + 1
    jsr     OVERLAY_ADDRESS
    pla
    jmp     loadCode

The function showCantAddPages is implemented on overlay 3. Code in overlay 2 can call _showCantAddPages in the record 0 code, which will load overlay 3, call the function, load the original overlay 2 again, and return.

Splitting the Logic

With helper functions in the record 0 code, it is possible to arbitrarily split logic into the different records. But since loading an overlay takes about 2-3 seconds on a 1541 disk drive, this should be minimized.

Central Library Code

The overlay code that we have seen above has to live in the record 0 code, so it’s directly callable by any record.

While in theory any other library code could live in any other record, keeping the most-used functionality in the record 0 code will reduce disk accesses. Here are some examples:

• generic dialogs
• error dialogs
• drive switching (app vs. document)
• disk full testing
• screen recovery
• font management
• zero page management
• some common text strings

One-time logic

Furthermore, there is code that is only ever needed once. On startup, the following is done:

• enumerate fonts and desk accessories on disk
• get the page size from the printer
• initialize the menu bar
• draw the ruler and the page indicator
• prepare a file opened for printing only
• do the copy protection dance

All this code lives in record 1. It is loaded immediately after the app is started. When it returns, it never gets loaded again.

Main Mode Code

Then, there is code that is needed when the app is in its main mode, like the text renderer and the handlers for navigating on the page, typing text and deleting text.

The main mode code lives in the remainder of record 0 as well as in record 2: During normal text editing, geoWrite always keeps record 2 loaded.

Since functions for menu items, keyboard shortcuts and mouse triggers in main mode are called by the GEOS KERNAL directly, which does not know about banking, at least the entry points of these handlers also have to live in record 0 (or, with restrictions, record 2).

Grouped Functionality

The remaining records contain further functionality, grouped by topic, so they can use common code inside the same record. Here is the list again:

• 3 cut, copy, paste
• 4 ruler editing
• 5 startup/about, create, open, paste text, run desk accessory
• 6 navigation, search/replace, header/footer, reflow
• 7 printing

On app launch, the record 0 entry code immediately loads initialization code in record 1. After its return, the record 0 code runs the startup UI code in record 5, which creates a new file or opens an existing file and returns to the core text editor in the record 0/2 code.

Duplicate Code

Common code needed by very different functionality groups usually lives in record 0, so that any record can call it without causing swapping in and out overlays. This is true for most common code, but since space in record 0 is at a premium, a different solution was necessary especially for bigger reusable components.

The startup code for example needs to talk to the printer driver to query the page size, and the print code needs to talk to the printer for printing. They both need to look up and load the printer driver. This common code is too big to fit into record 0, and having the startup code (#1) call out into the printing record (#7) would increase startup time by at least 5 seconds, swapping in #7 and then swapping in #1 again.

The solution is to just duplicate the common code into different records, e.g. by using .include statements to reuse the same code in multiple places. As long as the individual records don’t overflow their 4 KB maximum, this is a reasonable tradeoff.

Other examples are the document file version check, the string-to-int conversion code and common “text scrap” (clipboard) code. Parts of the latter are even included in three different records.

Conclusion

The overlay system has been a feature since the very first version of GEOS and is used by all major applications. It is not without its limits though: An application’s main mode should be responsive and do most of its work without swapping overlays, so the core logic (record 0 and one overlay record) has a certain limit in complexity. The individual overlays have a very tight size limit as well.

For the printing functionality, geoWrite already has to work around the overlay code size, which shows that an app like geoWrite is truly at the limit of what a GEOS application can do on a 64 KB system.