Aaru Progress Report: From March 2022 to November 2022

Welcome to the second progress report of 2022.

Ok, ok I'm not going to repeat it, yes I know it shall be once every month, but it has been a complicated year, so enjoy this year progress report.

AaruFormat V2

We have made great progress.

First of all we have a stable specification, in PDF, of the first version of AaruFormat. We do not suggest anyone implementing it as we're working on V2 but, it's there, if you want to peek at it.

We have clarified some of the more confusing parts of the implementation, so it should be easy to follow.

In V2 draft, we have defined structures to represent flux transitions as well as bitstreams, the kind of needed by the lowest level of floppy disk dumps, with hardware like the SuperCardPro or the Pauline board.

We also defined a completely new, multi-level based, deduplication table, that would allow us to store arbitrarily big media (without the problems that media bigger than 128GiB exhibited on AaruFormat V1).

This deduplication table also allows us to store twin sectors, as well as represent some copy protections, including but not limited to, sectors that return an error, sectors that return random data, sectors known to be physically destroyed ("holed") in the media, etc.

And our C implementation, libaaruformat, shall be completely able to read V1 images.

Next steps... Implementing the V2 specification in libaaruformat.

Linux and ATA/ATAPI shenanigans

We have discovered that a recent change in the Linux kernel implementation of ATA/ATAPI drivers, libata, they changed how they react to implicitly sized commands.

On the ATA/ATAPI specification there are some commands that are known to return a specific amount of data, usually 512 bytes (the original sector size from the first specifications). At the hardware level, you know when to keep reading data, because the drive is still telling you to do so.

This includes commands like IDENTIFY DEVICE and IDENTIFY PACKET DEVICE where you do not set a sector count on the drive registers, it will always return 512 bytes.

However recently Linux has been rejecting those commands back, indicating bad command parameters, and the solution is to tell Linux to expect to read one sector back from the driver.

We do not know when this was changed, but Windows seems to work as before.

A fix has been introduced in 5.3.2-lts and 6.0.0-alpha9 that shall be released next month, December 2022.

Please note that dumping CD/DVD/BD discs have not been affected, but dumping ATA or SATA disks as well as doing device reports is indeed affected.

Any device report that has been sent affected by this Linux change will need to be repeated, sorry for the inconveniences this may cause.

C# 11

This does not really affect users, but we have embraced the new C# 11 syntax and are changing our whole codebase to use it.

The result is that for any collaboration, things appear clearer and easier to change.

.NET 7

Microsoft recently released .NET 7.

And it is so much faster.

Between the performances improvements we made as explained in the previous reports, and the ones coming for free from .NET 7, we get a constant 20% to 30% speed-up in almost all operations with images!

And we plan to get even more speed, with the same quality, using new features from .NET 7 like .NET Native and single file publishing.

Some other things

• Dumping CD-Rs without getting errors on the run-out sectors at the end of the media issue #620 has been fixed.
• We added a workaround for drives that return track type errors too soon - issue #711
• Project files have been cleaned from archaic configurations, making Visual Studio Code and Visual Studio for Mac work better with our solution.
• When dumping CDs, INDEX 1 and ISRC would no longer be overwritten by wrong values from misaligned subchannel sectors.
• Fixed extracting files that contain illegal characters in Windows - issue #646

Translation

We have prepared Aaru for translation.

Our intention is to have as many translations for 6.0 as we can get, and for that, we need your help, a lot of help, because you will need to translate more than 9711 sentences.

There is time, before we release, to do it, so if you speak any language besides English or Spanish, your help on translation would be greatly, GREATLY, and I cannot emphasize it enough, GREATLY appreciated.

Community

Some people do not seem to know it, but we have three community places, where you can drop and speak to our team.

The first and more classic of it, is just plain IRC, in the Libera network, #aaru channel, you can connect using your favorite IRC client or directly using their web client.

Second, and quite alive, is our #aaru channel in the Video Game Preservation Collective. You can join there by just clicking here.

Lastly but not least, we have our own Discord server. It is designed for museums, preservationists, big collectors, institutions, NPOs, archives, libraries, etc. So if you belong to one of those groups and want to join our server, drop us a message. Developers that have presented a PR are also welcome.

Future

Holiday is coming!

We have some pressing bugs to fix, and we intend to release those fixes for 5.3.2-lts in December.

Besides that we will continue working in AaruFormat V2, implementing it, and changing the specification as the implementation tells us we need to improve.

And any help, translating, testing, and developing, Aaru and its components, is very much welcome.

Have a great month ahead.