Structure and Interpretation of Computer Programmers

I make it easier and faster for you to write high-quality software.

Friday, November 13, 2020

Apple Silicon, Xeon Phi, and Amigas

The new M1 chip in the new Macs has 8-16GB of DRAM on the package, just like many mobile phones or single-board computers. But unlike many desktop, laptop or workstation computers (there are exceptions). In the first tranche of Macs using the chip, that’s all the addressable RAM they have (i.e. ignoring caches), just like many mobile phones or single-board computers. But what happens when they move the Apple Silicon chips up the scale, to computers like the iMac or Mac Pro?

It’s possible that these models would have a few GB of memory on-package and access to memory modules connected via a conventional controller, for example DDR4 RAM. They almost certainly would if you could deploy multiple M1 (or successor) packages on a single system. Such a Mac would be a non-uniform memory access architecture (NUMA), which (depending on how it’s configured) has implications for how software can be designed to best make use of the memory.

NUMA computing is of course not new. If you have a computer with a CPU and a discrete graphics processor, you have a NUMA computer: the GPU has access to RAM that the CPU doesn’t, and vice versa. Running GPU code involves copying data from CPU-memory to GPU-memory, doing GPU stuff, then copying the result from GPU-memory to CPU-memory.

A hypothetical NUMA-because-Apple-Silicon Mac would not be like that. The GPU shares access to the integrated RAM with the CPU, a little like an Amiga. The situation on Amiga was that there was “chip RAM” (which both the CPU and graphics and other peripheral chips could access), and “fast RAM” (only available to the CPU). The fast RAM was faster because the CPU didn’t have to wait for the coprocessors to use it, whereas they had to take turns accessing the chip RAM. Nonetheless, the CPU had access to all the RAM, and programmers had to tell `AllocMem` whether they wanted to use chip RAM, fast RAM, or didn’t care.

A NUMA Mac would not be like that, either. It would share the property that there’s a subset of the RAM available for sharing with the GPU, but this memory would be faster than the off-chip memory because of the closer integration and lack of (relatively) long communication bus. Apple has described the integrated RAM as “high bandwidth”, which probably means multiple access channels.

A better and more recently analogy to this setup is Intel’s discontinued supercomputer chip, Knight’s Landing (marketed as Xeon Phi). Like the M1, this chip has 16GB of on-die high bandwidth memory. Like my hypothetical Mac Pro, it can also access external memory modules. Unlike the M1, it has 64 or 72 identical cores rather than 4 big and 4 little cores.

There are three ways to configure a Xeon Phi computer. You can not use any external memory, and the CPU entirely uses its on-package RAM. You can use a cache mode, where the software only “sees” the external memory and the high-bandwidth RAM is used as a cache. Or you can go full NUMA, where programmers have to explicitly request memory in the high-bandwidth region to access it, like with the Amiga allocator.

People rarely go full NUMA. It’s hard to work out what split of allocations between the high-bandwidth and regular RAM yields best performance, so people tend to just run with cached mode and hope that’s faster than not having any on-package memory at all.

And that makes me think that a Mac would either not go full NUMA, or would not have public API for it. Maybe Apple would let the kernel and some OS processes have exclusive access to the on-package RAM, but even that seems overly complex (particularly where you have more than one M1 in a computer, so you need to specify core affinity for your memory allocations in addition to memory type). My guess is that an early workstation Mac with 16GB of M1 RAM and 64GB of DDR4 RAM would look like it has 64GB of RAM, with the on-package memory used for the GPU and as cache. NUMA APIs, if they come at all, would come later.

posted by Graham at 09:35  

Wednesday, September 9, 2020

Dos Amigans

Tomorrow evening (for me; 1800UTC on 10th Sept) Steven R. Baker and I will twitch-stream our journey learning how to write Amiga software. Check out dosamigans.tv!

posted by Graham at 21:19  

Friday, August 7, 2020

grotag

Lots of Amiga documentation was in the AmigaGuide format. These are simple ASCII documents with some rudimentary markup to turn them into hypertext, working something like TeXInfo manuals. Think more like a markdown-enabled Gopher than the web though: you can link out to an image, video, or any other media (you could once you had AmigaOS 3, anyway) but you can’t display it inline.

Unfortunately choices for modern readers are somewhat limited. Many links are only now found on the Internet Archive, and many of those don’t go to downloads you can actually download. I found a link to an old grotag binary, but it was PowerPC-only.

…and it was on Sourceforge, so I cloned the project and updated the build. I haven’t created a new package yet, but it runs well enough out of Idea. I need to work out how you package Java Swing apps, then do that. It’ll be worth fixing a couple of deprecations, putting assets like the CSS file in the JAR, and maybe learning enough JavaFX to port the UI:

grotag AmigaGuide viewer

To use it, alongside your Guide file you also need a grotag.xml that maps Amiga volume links onto local filesystem paths, so that grotag can find nodes linked to other files. There’s an example of one in the git repo.

posted by Graham at 21:26  

Sunday, August 2, 2020

Getting started on my Vampire V4

Apollo accelerators make the Vampire, the fastest Motorola 680×0-compatible accelerators for Amiga around. Actually, they claim that with the Sheepsaver emulator to trap ROM calls, it’s the fastest m68k-compatible Mac around too.

The Vampire Standalone V4 is basically that accelerator, without the hassle of attaching it to an Amiga. They replicated the whole chipset in FPGA, and ship with the AROS ROMs and OS for an open-source equivalent to the real Amiga experience.

I had a little bit of trouble setting mine up (this is not surprising, as they’re very early in development of the standalone variant and are iterating quickly). Here’s what I found, much of it from advice gratefully received from the team in the Apollo Slack. I replicate it here to make it easier to discover.

You absolutely want to stick to a supported keyboard and mouse, I ended up getting the cheapest compatible from Amazon for about £20. You need to connect the mouse to the USB port next to the DB-9 sockets, and the keyboard to the other one. On boot, you’ll need to unplug and replug the mouse to get the pointer to work.

The Vampire wiki has a very scary-looking page about SD cards. You don’t need to worry about any of that with the AROS image shipped on the V4. Insert your SD card, then in the CLI type:

mount sd0:

When you’re done:

assign sd0: dismount
assign sd0: remove

The last two are the commands to unmount and eject the disk in AmigaDOS. Unfortunately I currently find that while dismounting works, removing doesn’t; and then subsequent attempts to re-mount sd0: also fail. I don’t know if this is a bug or if I’m holding it wrong.

The CF card with the bootable AROS image has two partitions, System: and Work:. These take up around 200MB, which means you’ve got a lot of unused space on the CF card. To access it, you should get the fixhddsize tool. UnLHA it, run it, enter ata.device as your device, and let it fix things for you.

Now launch System:Tools/HDToolBox. And click “Add Entry”. In the Devices dialog, enter ata.device. Now click that device in the “Changed Name” list, then double-click on the entry that appears (for me, it’s SDCFXS-0 32G...). You’ll see two entries, UDH0: (that’s your System: partition) and UDH1: (Work:). Add an entry here, selecting the unused space. When you’ve done that, save changes, close HDToolBox, and reboot. You’ll see your new drive appear in Workbench, as something like UDH2:NDOS. Right click that, choose Format, then Quick Format. Boom.

My last tip is that AROS doesn’t launch the IP stack by default. If you want networking, go to System:Prefs/Network, choose net0 and click Save. Optionally, enable “Start networking during system boot”.

posted by Graham at 23:55  

Thursday, July 30, 2020

The Nineteen Nineties

I’ve been playing a lot of CD32, and would just like to mention how gloriously 90s it is. This is the startup chime. For comparison, the Interstellar News chime from Babylon 5.

Sure beats these.

posted by Graham at 21:49  

Saturday, May 30, 2020

Continuous Integration for Amiga

Amiga-Smalltalk now has continuous integration, I don’t know if it’s the first Amiga program ever to have CI but definitely the first I know of. Let me tell you about it.

I’ve long been using AROS, the AROS Research Operating System (formerly the A stood for Amiga) as a convenient place to (manually) test Amiga-Smalltalk. AROS will boot natively on PC but can also be “hosted” as a user-space process on Linux, Windows or macOS. So it’s handy to build a program like Amiga-Smalltalk in the AROS source tree, then launch AROS and check that my program works properly. Because AROS is source compatible with Amiga OS (and binary compatible too, on m68k), I can be confident that things work on real Amigas.

My original plan for Amiga-Smalltalk was to build a Docker image containing AROS, add my test program to S:User-startup (the script on Amiga that runs at the end of the OS boot sequence), then look to see how it fared. But when I discussed it on the aros-exec forums, AROS developer deadwood had a better idea.

He’s created AxRuntime, a library that lets Linux processes access the AROS APIs directly without having to be hosted in AROS as a sub-OS. So that’s what I’m using. You can look at my Github workflow to see how it works, but in a nutshell:

  1. check out source.
  2. install libaxrt. I’ve checked the packages in ./vendor (and a patched library, which fixes clean termination of the Amiga process) to avoid making network calls in my CI. The upstream source is deadwood’s repo.
  3. launch Xvfb. This lets the process run “headless” on the CI box.
  4. build and run ast_tests, my test runner. The Makefile shows how it’s compiled.

That’s it! All there is to running your Amiga binaries in CI.

posted by Graham at 20:48  

Friday, May 8, 2020

SICPers podcast episode 5

It lives! Kinda. Amiga-Smalltalk now runs on Amiga. Along the way I review The K&R book as a tutorial for C programming, mentioning my previous comparison to the Brad Cox and Bjarne Stroustrup books. I also find out how little I know “C”, it turns out I’ve been using GNU C for the last 20 years.

Thanks to Alan Francis for his part in my downfall.

posted by Graham at 12:55  

Friday, May 1, 2020

SICPers podcast episode 4

We’re back to Amiga-Smalltalk today, as the moment when it runs on a real Amiga inches closer. Listen here.

I think I’ve isolated all extraneous sound except the nearby motorway, which I can’t do much about. I hope the experience is better!

posted by Graham at 08:53  

Saturday, January 6, 2018

MUI

In my last post I talked about investigating AROS, the modern, open source[*] implementation of the Amiga Operating System. Today I’ve spent some more time on that study, and found some things:

  • my strategy last time was to read the AROS application developer guide in order, up to the part about Intuition (the UI toolkit). This taught me about Windows and Gadgets, and the message-passing event system IDCMP. However, one of the first things I discovered today on digging more into that is that people don’t use these things directly.
  • The AmigaOS Documentation eschews gadtools (the library for working with Intuition gadgets) in favour of BOOPSI, the Basic Object-Oriented Programming System for Intuition. Its documentation uses demos that only work on the latest (proprietary) AmigaOS, and not on earlier AmigaOS or AROS.
  • The AROS documentation eventually led me to discover that even gadgets in BOOPSI are old and busted, and that the Zune[**] widget set for BOOPSI (which is based on the freeware Magic User Interface) is the new hotness.
  • This follow-your-nose-and-end-up-lost documentation issue is a big problem. I’ve learned a lot, but mostly follow “this is how this works” with “but you don’t do this”. Plenty of reference is made to the original Amiga developer documentation (which, luckily, I have, on CD in an Amiga-only format) and the MUI docs (which I don’t have but can probably get). This is not an AROS-specific issue, wherever “open source version of X” is documented, the documentation is likely to say “check out the documentation for X”. This only works where the documentation for X is available (imagine, say, if Commodore were to go bust in 1993 and stop publishing their developer docs), and relevant to the reimplementation (imagine, say, if Apple were to change programming language and GNUstep didn’t catch up).
  • That AROS is both source and (on m68k, or on UAE) binary compatible with programs that were written in the 1980s is amazing to me, and much more valuable than having latest whizzbang features and smartwatch integration from a modern platform. Leaving aside technical aesthetics which I’m likely to disagree with a lot of people on anyway, AROS and open source[*] systems like it (Haiku, ReactOS, FreeDOS, GNUstep, Lesstif) represent a critical piece of heritage infrastructure.

[*] I’m using this as a shorthand. AROS source code is published under the terms of the AROS Public License, which is not OSI-approved. The source code is available to use, study, share and improve, which many would understand to be “open”.

[**] Unrelated to the Microsoft Zune, and it’s unclear to me which came first.

posted by Graham at 23:26  

Sunday, December 31, 2017

AROS

I spent the weekend teaching myself some Amiga GUI (Intuition) programming using AROS via the Icaros Desktop distribution, their developer guides, and the Amiga developer CD. It’s a nice enough system to program in that works like most other GUI systems; GUI events are sent to your process as messages that you receive, handle, and reply to. GUIs are built out of gadgets attached to windows displayed on screens.

As a system to use, it feels efficient and fast. AROS is, as the Rhapsody developer releases were, a platform displaced in time: someone designed it to run on a 680×0 with a couple of megabytes of RAM and a floppy drive, and here we are with our modern CPU, gigabytes of RAM and solid state storage. It starts near-instantly, responds quickly (even though it’s running in VirtualBox under a Windows 10 host), it’s beautiful. And it’s not overloaded with widgets and gewgaws added by vendors who are keeping up with the Joneses by adding every new feature to every platform they support. No notifications ask me whether I’m interested in a tour of all the latest features that I’d be unable to find myself.

As I write this post (in the Odyssey Web Browser in Icaros Desktop, of course), I wonder whether I need all of the other things I would get if I were doing this directly in Windows, or in macOS, or a modern Linux distribution. Give me a compiler long enough and a place to save and I shall move the Earth.

posted by Graham at 16:07  
Next Page »

Powered by WordPress