o1i's Planet AROS

May 24, 2013

LallafaâBlog

plipbox 0.3 released

It took quite a while but now a shiny new release of plibbox is available: The new version 0.3 is an AVR firmware and Amiga driver update and works on the same hardware as presented in 0.1 and 0.2.

The major change in this release was motivated by Nitz76, one of my commenters in the blog: plipbox firmware now implements a MAC bridge and no IP NAT gateway anymore (see this technical introduction for details on the differences). In short: While with the old firmware you created a point-to-point IP link from the Amiga to your plipbox and from there the traffic was NATed and routed to your local ethernet, the new approach directly presents an Ethernet card on the Amiga side and passes the Ethernet frames generated from the Amiga TCP/IP stack directly via PLIP to your local Ethernet.

This approach is much more compatible in your local network infrastructure as the NAT is avoided. The change required a larger rewrite of the SANA II driver and therefore I heavily modified the original MagPLIP implementation and renamed the new driver to plipbox.device (the new source code is now also hosted in the plipbox source repository). Please see the updated Amiga Setup guide for installing this driver on the popular TCP/IP stacks including AmiTCP, Genesis, MiamiDX, and Roadshow.

The new MAC bridge made a lot of the TCP/IP code of plipbox obsolete and allowed me to shrink the firmware size considerably. Furthermore, setting up the firmware is zero-conf now, i.e. you don’t have to adjust any parameters for default operation.

This release now hosts all the docs on GitHub right next to the source. It is more convenient there to keep the docs in sync with the implementation and writing markdown is really fun! Another goodie in this release is the plipbox Emulator written in Python that allows to emulate a plipbox setup in FS-UAE. Its esoteric but really helpful for development: see my previous log post for details…

Enjoy!

by lallafa at May 24, 2013 07:30 AM

May 22, 2013

LallafaâBlog

Emulating plipbox with FS-UAE

Recently, I had some time to spend and wanted to work on the plipbox project’s Amiga driver. Unfortunately, I only had my MacBook Pro with me and no Amiga or plipbox hardware.

What does a SW developer do if the real hardware is not at hand?

Yes, you write an emulator that represents the missing HW in SW on a machine you have access to…

Adding a Virtual Parallel Port Protocol in FS-UAE

That said, I installed all required Amiga SW in a virtual environment in FS-UAE on my Mac and soon I had my plipbox.device SANA II driver talking with the virtual parallel port in FS-UAE. What was missing now is a protocol that communicates the complete I/O state from FS-UAE to another process. Then I could write a standalone application that does emulate the plipbox firmware and plipbox.device in FS-UAE would be operational.

I defined a simple two-byte bi-directional protocol called vpar that communicates any change in the emulation of FS-UAE’s Amiga parallel port to an external process and also receives external changes for input lines from the process and realizes them in FS-UAE’s emulated parallel port.

Having a POSIX compatible system with pseudo-terminals (ptys) I devised the link between both processes as a simple file-open interface in FS-UAE and a pty in the plipbox emulator (Note: any other interprocess mechanism works here also but this one was very easy to add in FS-UAE).

The setup looks like this:

        +----------+                                  +---------+
        | plipbox  |                                  | patched |
        | emulator |--(opens)--> [PTY] <--(file I/O)--| FS-UAE  |
        +----------+                                  +---------+
                    <--------- vpar protocol -------->

All required changes to FS-UAE are available in my clone of the FS-UAE repository on GitHub in the `chris-devel` branch.

For the plipbox emulator application I chose my favorite script language Python. With it setting up the pty for inter-process communication and implementing the vpar protocol peer was done in a few hours.

Setting up the Network

Ok, the plipbox emulator in Python can now talk PLIP via vpar’s virtual parallel port. The next step is to add access to an Ethernet device on the lowest frame level for our plipbox emulator.

I had a look at the pcap library that allows injecting packets into an existing adapter, but the Python bindings often lacked the inject feature of the library and so I dropped this idea.

Then I found a setup using a TAP device to easily create and receive ethernet packets from a user space application. Combine this with a real ethernet adapter via an interface bridge and you have access to real ethernet with a simple TAP file I/O interface (opening /dev/tap0) from your application.

The setup looks like this:

        +----------+              +---------+                +----------+
        | Real Eth |              | TAP     |                | plipbox  |
        | Adapter  |<-- Bridge -->| Adapter |<-- File I/O -->| emulator |
        +----------+              +---------+                +----------+
           en1         bridge0       tap0
           +----- OS net interfaces ----+

For the real ethernet adapter I use a second USB ethernet adapter on my Mac that is not configured (IP: 0.0.0.0) but active (up) in Mac OS X. This way OS X does not use it and my emulator can use it exclusively.

Creating the TAP device and building the bridge with the real adapter is done with some sudo’ed system command right in the startup of the emulator.

Note: the TAP driver is not shipped with Mac OS X itself, but you can add it by installing the ones from the TUN TAP OSX project.

Putting it all together

On top of the vpar Python module I implemented the magPLIP protocol layer in an own Python class. Now all pieces to create the plipbox were available.

The plipbox main script sets up the vpar, magplip, and ethernet with TAP and interface bridge. In the main loop it does I/O multi-plexing with select() on the file descriptor for the vpar PTY and for the TAP file.

A packet arriving from TAP is filtered (i.e. multicast and unknown broadcasts are removed) similar to the real plipbox firmware and then delivered to the emulated Amiga via the vpar connection.

A packet incoming via PLIP is directly delivered to the local network and thus written to the TAP file.

That’s it! A working plipbox running as a Python script on my Mac… Similar to the firmware I added some optional debugging output to see the packet contents and to measure the timing and latency. Now I can develop and test the plipbox.device driver with the emulated environment…

Sneak a Peek?

The plipbox emulator will be shipped with the upcoming version 0.3 of plipbox. If you want to have a look already then head over to my GitHub repository where all development takes place… Have a look in the python and doc dirs :)

 

by lallafa at May 22, 2013 07:02 PM

May 19, 2013

E-UAE PPC JIT

One small step for mankind, a giant leap for the project

I have no idea how did I manage to achieve this much in this update, but it is certainly a confident step forward. For this time the list is long and diversified:
  • Implementation of Bcc.x addr, BCHG.B Dx,mem, BCHG.L Dx,Dy, BCLR.B Dx,mem, BCLR.L Dx,Dy, BRA.x abs, BSET.B Dx,mem, BSET.L Dx,Dy, BSR.x abs, BTST.B #imm,mem, BTST.B Dx,#imm, BTST.B Dx,mem, BTST.L Dx,Dy, CMP.x #imm,mem, CMP.x mem,Dy, CMP.x reg,Dy, CMPA.L reg,Ax, CMPA.W reg,Ax, CMPA.x mem,Ay, DBF.W Dx,addr, EOR.x #imm,mem, JMP.L abs, JMP.L mem, JSR.L abs, JSR.L mem, NEG.x mem, NOT.x mem, RTS, TAS.B Dx, TAS.B mem instructions.
  • Cache invalidation fix for OSX 10.3.9 and below. (Thanks to Mike Blackburn again.)
  • Fixed mask handling in BCHG.B Dx,mem instruction.
  • Fixed missing register mapping in ASL.x #imm,Dy implementation.
  • Fixed input dependency overwriting in certain memory-related allocation functions.
  • Fixed dependency for destination memory pointer register in special memory reading.
  • Fixed post address handler for condition code addressing modes, previously it might crash or call some random handler from the other addressing modes.
  • Fixed instructions where temporary registers are allocated but not free'ed.
  • Optimized masking for register to register bit instruction.
  • Optimized the temporary register usage in helper_test_bit_register_register function.
  • Optimized flag extraction in several shifting operation.
  • Branch scheduling is more flexible: adding multiple interleaved branches is possible.
  • Comment on missing implementation for an exception on loading odd address into PC.

A few highlights

First of all, let me brag around a little bit about the number of freshly implemented instructions. Right now 237 instructions are implemented out of 388, a solid 61% is done. (Previously the ratio was ~46%.)

More MacOSX versions are supported now, Mike fixed up the cache flushing a little bit and added the pre-10.4 versions too. Please read the included README file regarding the compiling instructions.

While I was working on the instructions I discovered a few bugs and glitches, which are now fixed in this release thus improving the overall stability.

I have also managed to optimize the compiled code for some instructions. Together with the implementation of some yet missing instructions the results for the Mandelbrot test (mandel_though_hw.kick.gz among the test kick files) improved a bit compared to the previous results:

Interpretive: 108 seconds (no change there...);
JIT compiled without optimization: 44 seconds (previously it was 52 seconds);
JIT compiled with optimization: 27 seconds (previously it was 32 seconds).

That was the time for the self-polishing and now back to work...

by noreply@blogger.com (Álmos Rajnai) at May 19, 2013 08:57 AM

May 09, 2013

Icaros Desktop

Icaros Desktop 1.5.2 now ready for download

Enjoy! A new, updated version of Icaros Desktop is finally here! Brand new system files fixed many bugs and quirks of the underlying operating system, while many little fixed and additions will improve user experience with everydays use of the distribution. Both stability and compatibility should have improved, and with the addition of new games and emulators like Pushover and SuperModel, our

by Paolo Besser (noreply@blogger.com) at May 09, 2013 01:32 AM

May 08, 2013

AROS-Exec

Icaros Desktop 1.5.2

New version 1.5.2 of AROS distribution Icaros Desktop has been released. The update brings fresh system files, some new applications and many bug fixes, improving speed, stability and compatibility. A more complete list of improvements is available, as usual, on the project's web page, http://www.icarosdesktop.org

Original Image- Original Image

May 08, 2013 09:40 PM

Icaros Desktop

Really silly bug found in LiveUpdater

Hello to everyone! First the good news: Icaros Desktop 1.5.2 is ready. But I think I will announce it in a better fashion later. Then, the curious one: I've found a little quirk in LiveUpdater that, strangely enough, no one has ever reported to me. In practice, instead of selectively update contents in Extras: depending on your first installation choices, it copies everything it finds in the

by Paolo Besser (noreply@blogger.com) at May 08, 2013 06:29 PM

April 28, 2013

E-UAE PPC JIT

Locations, locations, locations

In the last month I was trying to hang on my sanity while we were on house-hunting in Auckland (nuff' said). Wasn't easy and apparently it is not even close to be finished. :/

Anyway, I managed to do some work on the E-UAE JIT in the stolen moments.

In the update for this month you will find these little eggs:
  • Implementation of ADD.x Dy,mem, ADD.x mem,Dy, ADD.x #imm,mem, ADDA.x mem,Ay, ADDQ.x #imm,mem,
    AND.x mem,Dy, AND.x reg,mem, ANDI.x #imm,mem,
    BCLR.B #imm,mem, BCHG.B #imm,mem, BCHG.L #imm,reg, BSET.B #imm,mem,
    NEG.x Dy,
    OR.x Dy,mem, OR.x mem,Dy, ORI.x #imm,mem,
    UNLK.x Ay instructions.
  • Fixed unintended modification of the source register for some register to memory operations.
     
  • Memory read helper tweaked to use R3 register as the result register, no need to copy the data back-and-forth. (More optimal compiled code.)
     
  • Memory reader and writer helper function cleaned up to be more independent from caller data.
It might seem a bit random how I choose which instructions are implemented, but there is always a recurring theme. Right now this theme was the memory access. As you can see most of these instructions are manipulating the memory, which was a little bit scary earlier but I came around creating some functions which can be reused for (almost) all memory accessing instructions.
The tricky part was accessing the memory while the allocated temporary registers remain accessible somehow. With a minor workaround for saving and occasionally reloading the temporary registers after the memory access this is solved now.

I am not too happy about how the whole register mapping works, unfortunately there are some limitations of the C language which makes it complicated to come up with a more robust solution. So, right now the whole thing is just a bit hacky and wacky. Maybe in the future it would need an overhaul.

I get the question most of the times: how many instructions are left to implement. There is an easy way to find out the progress: check the table68k_comp descriptor file.
Each (to be) supported instructions for the JIT compiling is already listed there, next to the name of the instruction there is a number: 0 or 1. The 1 means it is already done, 0 remains to be implemented.
The instructions which will not be supported by the JIT compiling (so the interpretive will handle these) are not listed in this file.

So, all we need to do is counting the instructions which are already supported and what remains to be done. The current state without the FPU instructions is: 181 is done out of 388 (~46% is done).
As you can see there is more work to do, but it is really hard to tell how long does it take. What I can see is that the time I have to spend with each instruction is shorter and shorter, due to the infrastructure which had to be built first but now it is mostly done. Also some instructions are very similar, I can simply reuse parts of an already finished instruction.

We are not there yet, but the donkey is not that stubborn anymore. Giddy-up buddy!

by noreply@blogger.com (Álmos Rajnai) at April 28, 2013 08:28 AM

April 24, 2013

E-UAE PPC JIT

Mac and cheese... err... Linux

Big thanks to Mike Blackburn for some fixes for the Macintosh support and for implementing the Linux PPC support! Well done, Mike.

After this (and this) update Mac OSX 10.4 is supported too: the instruction cache flush needed a different implementation.

Also Linux PPC users can benefit from the PowerPC JIT.

The more the merrier.

by noreply@blogger.com (Álmos Rajnai) at April 24, 2013 07:45 AM

April 22, 2013

Icaros Desktop

AROS Shell Manual updated again!

A good news for every AROS user: a new edition of the AROS Shell Manual is now available for download @The Archives. It was very long ago when Joao Ralha firstly collected commands documentation from Mathias Rustler (and other sources) into a full featured, cosy manual in PDF format, and the AROS Shell Manual has been a very good friend to shell scripts coders since then. With time, however,

by Paolo Besser (noreply@blogger.com) at April 22, 2013 11:15 AM

April 12, 2013

Icaros Desktop

AROS-2-PENDRIVE virtual machine

One of the most common questions about Icaros Desktop is: "I do not own a internal or external USB DVD-ROM drive, how can I create a installation pendrive?" and, frankly, it's not a trivial question to answer to. Other operating systems allow using programs like unetbootin to create a "installation pendrive" from ISO files, but this is not an option for Icaros Desktop, since AROS kernel does not

by Paolo Besser (noreply@blogger.com) at April 12, 2013 05:02 PM

April 03, 2013

Binarydoodles

Diary:Technologies: Sweet as a Raspberry Pi :9

First of All, Thanks to the ilCannocchiale staff for fixing comments; pity lots of more stuff need to be still fixed including permalinks...

This year had life going through and at the last moment prevented me to participate at SCaLE; might need to change home again, this time due to a problem with neighbors too long and complicate to discuss here; thank God the work side is good and have no complains.

Not too much news under the AROS front at first glance; however, as usually "look is deceiving" because we had a good amount of ports; Especially Slazard 'Bszili' Biro and Yannick 'Yannickescu' Erb are bringing to AROS a good amount of games and emulators i can remember Frogatto, TORCS (not released due to problems with GLSL), Flare, Gigalomania and lots of others; Joao Ralha, recently opened a new AROS blog called AROS Playground where reviews ported games and provides developer interviews;

Timothy 'Terminills' Deters is working on porting Gutenprint on AROS, on interfacing it with a generic Amiga printer driver.
Gutenprint, formerly know as Gimp Print are High Quality printer drivers based off the hpijs printer system; the name is from the fact that those drivers were originally developed for GIMP; were also adopted from Mac os X in the early version as printer drivers.
According to its plans Gutenprint will talk to the printer via a bridge software and to a generic 3.1 driver written by Jason 'Ezrec' McMullan; the driver written by Jason is also able to translate Gutenprint commands in Turboprint and 3.1 printer commands, however this implementation is not limited to Gutenprint; the printer driver is a 24 bit one and in the intentions of Terminills should also bring advanced features such as ipp (internet Printing Protocol), printer detection and driver auto detection; Terminills is also working at the zune based interfaces to configure the printers;

Among all the ports of Yannick 'Yannickescu' Erb, in 2010 was the long forgotten and neglected structured Graphics program Amifig to AROS; started a slow but steady work on updating it first improving the rendering, then adding file export and import with a side application; now a new version is available, with a new name, ZuneFig, a rewamped User Interface now in Zune, support for gradient and  Library previews and a better redraw with Anti-Alias;

Nik 'Kalamatee' Andrews is working on the native Raspberry Pi port of AROS; on March 14 he has been able to make AROS boot to wanderer;  the pictures of the first boot have been spreaded on the net via twitter and Amiga and AROS forums and communities; last week Kalamatee worked on debugging the SD card driver so to support also SDHC; the SD card support is also actually under further development by Pavel 'Sonic' Fedic to bve implemented as multiplatform device driver since the logic is changing form platform to platform; right now Kalamatee loosened a bit the grip on the port since he need to implement USB and still is trying to find a way to implement it. Curious is also that among the motivations of Kalamatee is the idea to finish its modular overhaul of Wanderer so that could be optimized for a less powerful machine like the Raspberry PI.

To finish,lately on aw.Net some threads appeared where people seems interested in a new kernel for amiga os or its like tastes; is known that in the past at least two attempts have been made: one was the well known Anubis, brought on by Alex 'MrBIOS' Perez, the other one was ARIX, started by Michal Schulz and from which the idea of Anubis took shape; summarizing, the idea behind those two partial forks is to have the actual exec kernel substituted by a linux one together with a wrapper that translates the AROS API calls in linux ones so that AROS binaries libraries and programs can run on top of it with no recompilation; is a similar approach to the system adopted by Android and can bring the advantage of memory protection, SMP and linux drivers to AROS; both projects have an actual unknown status and would like to know by the maintainers if there has been any progress since.

by saimon69 at April 03, 2013 05:00 PM

April 02, 2013

o1i

p-uae v0.1

I decided to do a quick alpha release of my p-uae port, grab it from the aros-archives / aminet, as soon as is accepted there.

This alpha release of P-UAE for AROS is based on the sources at http://github.com/GnoStiC/PUAE from the 31st January 2013.

This is a *real* alpha version, don't expect anything: It *will* crash, it *will* miss features and so on.

As PUAE is based on the merged WinUAE version, it means, that P-UAE v0.1 is based on WinUAE 2.6.0.

I introduced a new version naming system for P-UAE for AROS, as I will improve this release and there is no other way to distinguish two releases of P-UAE 2.6.0 otherwise.

You can get the sources to this build from the p-uae branch of the janus-uae repository, online browse-able
here:

http://janus-uae.svn.sourceforge.net/viewvc/janus-uae/branches/p-uae/PUAE/

P-UAE reads the config from a .uaerc file, so if you have a j-uae configuration, copy/rename it.

What does *not* work, is the GUI. It will show up, but will be useless. So better keep the use_gui=no setting.

What seems to work:
- OCS/ECS
- keyboard, mouse
- harddisk emulation with filesystem
- floppy emulation
- 68020/68881 CPU emulation

Everything else, I have not yet tested ;).

I just wanted to give you something to play with. This is for the brave ones, there is no default .uaerc supplied.

Bug reports are maybe not yet useful. Let me know, if the above does not work for you at all. But I always like to hear, if something *does* work, of course ;).

by noreply@blogger.com (o1i) at April 02, 2013 12:11 PM

April 01, 2013

E-UAE PPC JIT

After another bump on the bumpy road

I have spent some time fixing bugs and improving the performance of the compiled code and the JIT emulation overall. As a result here is the recent update:
  • Implementation of TST.x mem, NOT.x Dy, EXT.x Dy
  • Reverted the change for stopping the compiled block when the special flag is set. This change is not needed for the Kickstart and it doesn't seem to have any effect, but slows down the compiled code.
  • Removed hack for MOVE.z Ax,-(Ax) and MOVE.z Ax,(Ax)+ instruction implementations.
  • Fixed pre-decrement and post-increment addressing modes for destination. Re-enabled MOVE.x mem,mem and EOR.x reg,mem instructions, which were disabled previously to let the Kickstart running without the addressing mode fixes.
  • Fixed register dependency in some memory addressing modes.
  • Temporarily removed checking for the tiny blocks, ignoring these blocks is just not the right solution for avoiding the overhead of block calling.
  • Added ignoring blocks of pure unsupported instructions: these will be executed by the interpretive emulation.
  • Fixed blocking of small blocks, the block was not raised in the cache list.
Lots of small changes and fixes as you can see.

Probably, the most important fix was the one for the pre-decrement and post-increment addressing modes, this was blocking the Kickstart for a while from booting and this is why I had to remove the support of those two instructions I mentioned in the changes list.
As it turned out the root of this bug was a limitation of the implementation. Each addressing mode has two compiler function: one is called before the instruction compiling, one is called after that. But the situation is not always that simple, for example in this case:

MOVE.L (A7),-(A7)

This is a common instruction for copying the memory to a lower address (like moving the content of an array one step toward the beginning, in this case the content of the stack). Seems so innocent, isn't it? :)
Why this one was an issue: the handler for the destination address is called before the instruction and since it is a destination addressing mode it decremented the address in the emulated A7 register. But then the instruction was compiled, which used the address from the A7 register as a source. So, what actually happened was something like this:

MOVE.L -(A7),(A7)

Now you can see: this operation is (mostly) pointless, it copies the data from the one address to the same address. (Although, sometimes it might make sense in the communication with the hardware, but this wasn't the case right now, obviously.)

What was the fix? Pretty simple: I moved the destination address modification to the address handler which is executed after the instruction was compiled. This was a solution for this specific case, but I also had to make sure that all the other combinations are working which might be possible with the indirect addressing. One of the trickiest was:

MOVE.L (Ax)+,(Ax)

You can try to guess why. :)

Anyway, finally this bug is out of the way and I can go on with implementing the missing instructions. Some of them are done, but yet lots to go.

There was one more important change in this update: I removed the limit for the consecutive block length when a special condition was triggered by some instruction. I found it completely pointless, everything seems to be working without this condition. There was a bad side-effect of this limit: after an instruction triggered a special condition for the emulation all the following instructions are compiled one-by-one into separate blocks. The overhead for calling these tiny blocks was huge, this is why I introduced the rule of ignoring any block which was smaller than 3 instructions. But in this case lots of the code was not JIT compiled at all. (As some of you guys mentioned: the JIT LED was mostly dark - lots of blocks were not compiled.) This is fixed now, although I am a bit afraid of the side-effect of delaying the handling of the special conditions. We will see how it goes.

I also spent some time on updating my old tool: DiskDaisy. The recent updates for AmigaOS4.1 triggered a bug in that app. Sometimes it is nice doing something completely different for a while, you know. :)

by noreply@blogger.com (Álmos Rajnai) at April 01, 2013 08:42 AM

March 28, 2013

o1i

.. and in 24 Bit

SDL can force a 32bit mode, so on a 24bit screen we do a:

SDL_SetVideoMode (current_width, current_height, 32, uiSDLVidModFlags);

And with this magic, PUAE also works in my 24-bit AROS/linux hosted environment :)!


And as you can see in the above picture, harddrive support is now also enabled, the above os3.1 was booted from a emulated harddrive.

There seems to be at least one strange bug still hidden somewhere, as the version command fails for no obvious reason. Seems to be a problem with loading files from the harddrives, but I am not sure.. Directory listings seem to work quite fine, though.

by noreply@blogger.com (o1i) at March 28, 2013 02:05 PM

March 27, 2013

o1i

PUAE .. in 16bit only ..

Debugging PUAE really sucked, as everything seemed to be working perfect, just the window stayed black..


But one simple trick made it working: switching my windows resolution to 16 bit, which forced my x11 server on windows to 16bit, which then caused AROS/hosted to 16 bit .. which made PUAE/AROS display the above window!

Why?

The showstopper seems to be gfxvidinfo.pixbytes, which is 2 for 16 bit and 4 for 32 bit..
but for AROS hosted on X11 I get 24 bit, which results in gfxvidinfo.pixbytes of 3(!).

Have a look at the UAE code (all UAEs!):

static void pfield_do_fill_line2 (int start, int stop, bool blank)
{
  switch (gfxvidinfo.pixbytes) {
  case 2: fill_line_16 (xlinebuffer, start, stop, blank); break;
  case 4: fill_line_32 (xlinebuffer, start, stop, blank); break;
  }
}


So you notice, what happens? 

If pixbytes is 3, *nothing* happens. No error, no warning, nada. 

And this is, why debugging sucked.

So at least now I know, what causes the problem .. the solution is still a bit away.

by noreply@blogger.com (o1i) at March 27, 2013 03:41 PM