Visualizing Commodore 1541 Disk Contents – Part 2: Errors

We have previously visualized the physical layout of C64/1541 disks. In order to understand the encoding and potential read errors, it is more useful to visualize the disks sector-by-sector.

This animation shows 12 regular disks. (Click for original size.)

• Every pack of 17-21 lines is a track, numbered 1-41.
• Every line within a pack is one sector.
• The raw sector contents are drawn from left to right.
• The cyan part is the header, the green part the data.
• Black is 0, cyan/green is 1.
• White represents missing header or sector sections.

The tool to generate these images from .G64 files is available at https://github.com/mist64/visualize_1541.

Header

Let us look at the header first. Here is an animation of the header of tracks 23 to 25 of a few disks:

The header contains six data bytes encoded using the 4-to-5 GCR scheme:

• H: Header Code: Every header starts with 0x08 so it can be distinguished from the sector data (0x07). You can see that it is the same bit pattern on all sectors on all tracks.
• T: Track: This is the track number (1-35). It is the same for all sectors on the same track.
• S: Sector: This is the sector number (0-16/17/18/20, depending on the track). You can see the same bit patterns on each of the three tracks shown.
• ID: ID: The two-byte ID should be unique per disk and is used to detect disk changes. It is the same for all sectors on the same disk.
• C: Checksum: The checksum is the XOR of C, S, T and ID, so you can see it behaves kind of randomly in the animation above.
• GAP: The gap does not contain usable data, but separates the header from the sector data.

Data

The first byte of the data section is the code 0x07 to distinguish it from a header. The rest is the 256 data bytes. The Commodore DOS filesystem uses two link bytes (next track, next sector) at the beginning of every sector, which is why the next two bytes in this visualization look more regular between disks.

Good Disks

First, let’s look at some error-free disks in practice.

Empty Disk

An empty disk looks quite uniform: (Click for original size.)

Only sectors 0 and 1 of track 18 look different:

18/0 contains the block allocation map (BAM) and the name of the disk, and 18/1 contains the first 8 directory entries.

Illegal Bits

The last few bits of the end of the header are also different for sectors 0 and 1. The headers of all sectors directly after formatting the disk end in a pattern of alternating zeros and ones, but as soon as a sector has been written to, this is no longer the case for the last few bits. It gets even more interesting when visualizing these bits across several reads of the same disk:

These are unstable illegal bits on disk, bits that sometimes read back as zeros and sometimes as ones.

When writing a sector, the software in the disk drive waits for the correct sector header to pass by the read head, then waits until the exact end of the header gap area, and starts writing the new data. When switching to write mode, the magnetization written onto the media for the first ~15 µs (4-5 bits) will be analog values between logical 0 and 1. These are technically illegal values, and will be unstable when read.

These illegal bits are benign, because they do not encode anything. But it is important to note that two reads of a good disk may not be identical.

Errors

Now, let’s look at what common errors look like:

Logical Errors

Some read errors are caused by buggy software writing to disk: (Click for original size.)

This disk is missing some sector headers:

This is one of the faulty GEOS boot disks: Some sectors were written with the wrong speed zone setting, so they spilled into the next header, overwriting it. The sector contents are still there, but the missing header makes them unreadable with the original Commodore DOS. (The visualization tool will guess the sector number in case of a sector with a missing header.)

Dropouts

There is an example of a dropout:

There is one sector that starts reading back as all 0 bits (black) at some point, and the next sector data is completely missing (white), which is because the SYNC mark of the next sector was not readable.

This can be caused by demagnetization, or more likely, by dirt on the disk’s surface. Sometimes, the dirt will scrape off if we just read the disk often enough. The following animation shows the result of subsequent reads of the same disk:

Here, you can see how the faulty sector on the third track in the picture starts out all-black, with the next sector missing, over unstable data reads, to finally stable and correct reads. You can also see how the previous two sectors are impacted by the same speck of dirt, just not as much.

(The fact that it is different sector numbers that are impacted on the different tracks is due to the fact that on 1541 disks, there is no sector alignment between tracks, i.e. whether sector 0 on one track touches sector 0..20 on the next track is practically random.)

If the dropout persists across retries, it might still be dirt, just of the more resilient kind. Just clean the disk and try again.

Weak Bit Data Errors

This is a disk with multiple checksum errors in the data section across several tracks. Cleaning the disk did not help, so these are weak bits:

Weak bits do not result in flipped bits, but in missing or duplicate bits. As you can see in the animation, sections of the data move back and forth between retries. It is clearer when looking at a single sector. This excerpt contains four weak bits:

The read logic of Commodore disk drives measures the time between zero-to-one and one-to-zero edges and then decides how many zeros or ones were on disk. Therefore, weak bits lead to duplicate or missing bits.

A single weak bit can be recovered from by retrying: If it reads back correctly every now and then, it is as easy as retrying until the checksum is correct. But with multiple weak bits in a single sector, this gets exponentially less likely.

Conclusion

All that current tools can do with faulty disks is retry until the checksum is correct. By analyzing the weak bits, it should be possible to create tools for data recovery on a lower level. Multiple reads will reveal where the weak bits are, so a tool could try out different sequences around the weak bits and verify the checksum.