Log in




Reset your TomTom

August 29th, 2011 by Peter

My TomTom starts to behave like a misconfigured, old Xserve with Tiger installation, so I decided to give him a fresh filesystem and software installation. It turns out that that this is harded than expected, but here is the ultimate guide:

http://de.support.tomtom.com/app/answers/detail/a_id/13599/kw/dateisystem

Apache 2.2 authentication against local password database

August 3rd, 2011 by Peter

You have this problem from time to time – how can you easily protect the access to a virtual host, based on the local user database ? For my Debian system, I found mod_authnz_external to be handy:

http://code.google.com/p/mod-auth-external/

Here is the list of steps that worked for Debian Squeeze:

  • apt-get install libapache2-mod-authnz-external
  • Enable it by linking /etc/apache2/mods-available/authnz_external.load to /etc/apache2/mods-enabled/
  • Add the following lines in the <Directory> or <Location> section of your virtual host:
    AuthType Basic
     AuthName "Area 51"
     Require user [youruser1] [youruser2]
     AuthBasicProvider external
     AuthExternal pwauth
  • Add the following lines on the level of the <VirtualHost> configuration:
    AddExternalAuth pwauth /usr/sbin/pwauth
     SetExternalAuthMethod pwauth pipe
  • Reload Apache, test, read the logs to find the problem …

Windows EFI support

February 8th, 2011 by Peter

For our work on an EFI-based fault injector, we researched the EFI / UEFI support in the Windows operating system family. Here are the results collected from an introduction document, a requirement specification , a presentation from the UEFI plug fest, another presentation on the web,  and the Windows Research Kernel documentation project.

Windows version

The support for EFI / UEFI for Windows began in 2002. It is only available for 64-bit processor architectures:

  • Windows XP (X64): Can at least boot from an EFI firmware, since the according NVRAM boot option manipulation tools are documented.
  • Windows Server 2003 (IA-64): EFI 1.10 support
  • Windows Server 2008 (IA-64): EFI 1.10 support
  • Windows Server 2008 (X64): UEFI 2.0 support
  • Windows Server 2008 R2: UEFI 2.3 ‘non-removable media boot behavior’ support
  • Vista SP1 (X64): UEFI 2.0 support
  • Windows 7 (X64): UEFI 2.0 support

There are only a few official remarks about the usefullness of EFI for Windows features beside booting. Microsoft claims ‘multicast deployment’ and ‘faster hibernate’ as examples. The standard system table with both boot services and runtime services is expected to be fully implemented by the firmware. Windows makes use of the EFI PXE support, the device path protocol, and the block I/O protocol at boot time and on hibernation resume. Furthermore, a built-in TPM module is expected to be usable through the according EFI protocols.

Windows always prefers native drivers over ACPI runtime support over EFI runtime support functions. The currently documented amount of EFI support fits to this policy.

Boot support

Windows officially switched to the firmware-independent Boot Configuration Data (BCD) approach since Windows Vista, which means that the Windows boot manager (bootmgfw.efi) has its own storage facility for boot options and ordering. The BCD store is located as file on the EFI partition. The boot loader EFI application uses the Block I/O protocol to read the file, and later to bring up the OS loader. In general, it seems like the BCD store and NVRAM variables for booting are kept in sync. If this is the case, the BCD editing during runtime *could* lead ultimately to calls into the UEFI runtime variable service – we still need to check that. The presentation from the plug fest claims GetVariable, SetVariable, and GetNextVariableName to be the only EFI functions used at runtime. Another document lists QueryVariableInfo as another function used at runtime. The purpose seems to be related to WHEA, we are still looking into this one too.

On boot time, Windows is able to use either the EFI 1.10 universal graphics adapter protocol (UGA), the UEFI 2.0 graphics output protocol (GOP), or the standard simple text output protocol.  Windows Server 2008 R2 and Windows 7 show a high-resolution version of the startup animation if GOP is supported.

Runtime services (Updated)

The investigated documentation underlines the impression that EFI runtime services are respected and protected by the Windows operating system, including the support for EFI’s GetMemoryMap(). The neccessary ExitBootServices() and SetVirtualAddressMap() calls are performed by the Windows OS loader, which runs after the boot manager got a choice for the OS installation to be started.

A representation of EFI runtime services in Linux is given by it’s efivars module. For Windows, our crawling of MSDN finally lead to the well-hidden GetFirmwareEnvironmentVariable and SetFirmwareEnvironmentVariable functions. Disassembling their implementation on Windows XP 32-bit showed that GetFirmwareEnvironmentVariable maps to NtQuerySystemEnvironmentValueEx, and SetFirmwareEnvironmentVariable maps to NtSetSystemEnvironmentVariableEx. Both kernel functions are declared in NtExApi.h and implemented as part of sysenv.obj in a Windows kernel build, making them ultimately a part of ntoskrnl.exe.

In the Windows XP 32-bit case, both kernel functions map to the same implementation that just returns STATUS_NOT_IMPLEMENTED.  This fits to the MSDN documentation, which declares support for the user mode functions starting from Windows XP SP1 – but this relates only to the export of the according symbols in kernel32.lib. Therefore, the EFI support levels described in the first part of this article still apply. Applications need to check (with the procedure described for GetFirmwareEnvironmentVariable) if the functions are implemented.

We are currently about to disassemble the Windows kernel behavior on a true EFI system, in order to get a better feeling of the call flow when the implementation is there.

Building PyV8 on Debian / Ubuntu

November 30th, 2010 by Peter

PyV8 is hard to build from scratch, but the majority of current platforms (recent Ubuntu versions, Mac OS X Snow Leopard) cannot make use of the officially provided binaries. Here are the building instructions for Ubuntu 10.10 on 32bit systems, which comes with Python 2.6 as default.

  • Install relevant software, for example on Debian:
    • apt-get install gcc g++ bjam scons subversion python-dev libbz2-dev
  • Get all the relevant source code. The PyV8 wrapper relies on Boost, V8 is self-contained. Resist the temptation to use the source packages from your distribution, since the (sub-optimal) design of the PyV8 build script cannot deal with modified paths and file locations.
    • svn co http://pyv8.googlecode.com/svn/trunk/ pyv8
    • svn co http://v8.googlecode.com/svn/trunk/ v8
    • svn co http://svn.boost.org/svn/boost/trunk/ boost
  • Build V8:
    • Call “scons -c .;scons” in the V8 source root directory. By default, “-Werror” is used to translate all GCC warnings into errors. I ran into problems with newer GCC versions because of this. If you know what you do, you can remove the “-Werror” flag from the according part of the SConstruct configuration file.
    • The default setting is to generate the release version of the static library, which is exactly what we want.
  • Build and install Boost (or more specifically, the Python Boost libraries):
    • Move to the Boost source code directory.
    • Call “./bootstrap.sh”, “bjam -a -q” and “bjam install”
  • Change to the PyV8 root directory, build it and install it:
    • export V8_HOME=../v8
    • export BOOST_HOME=../boost
    • python setup.py build
    • sudo sh -c "export V8_HOME=../v8;python setup.py install"

I/O Performance Analysis in Linux

November 25th, 2010 by Peter

If you want to analyze the I/O performance behavior of your system, there are several tools available. “vmstat” is the classic, and has a “-d” option that gives you a summary of devices and their raw activities. A similar purpose is fulfilled by “iostat -d”, which computes additional I/O rate informations. The third tool is “iotop”, which shows you a process list with according I/O activity information.

TCP/UDP ports used by MacOS X Server

November 23rd, 2010 by Peter

We faced the question of the ports that must be opened in the firewall for some of the Mac OS X Snow Leopard Server daemons – iChat server, iCal server, Adressbook server, …

The following document contains a list of all relevant port numbers, just start to read from page 114:

http://manuals.info.apple.com/en_US/NetworkSvcs_v10.6.pdf

Another good summary is in the knowledge base:

http://support.apple.com/kb/ts1629

For other server version, search the Apple support site for “network services administration”.

STRACE for Mac OS X

November 15th, 2010 by Peter

MacOS X does not have native support for the well-known Linux tool strace, which allows you to monitor system calls (and their parameters) for any running process.

Until MacOS X Leopard, the typical recommendation was to use a combination of the tools “ktrace” and “kdump”. Leopard introduced support for DTrace, so both tools were removed in favour of D-Trace based monitoring scripts.The problem is that you often have an ‘emergency’ debugging situation, were you don’t want to learn the D-scripting language for one single purpose.

Luckily, there is a set of D-scripts already provided by a standard MacOS X installation:

man -k dtrace

One of the default scripts coming with your operating system installation is “dtruss”, which is the closest thing to strace you can get.

Official Debian repository for Condor available

July 28th, 2010 by Peter

6 years (!) after my initial Debian ITP (see here) and all the hard work on a custom Condor Debian package, they finally made it too:

http://www.cs.wisc.edu/condor/debian/

The Condor people are now offering there own Debian repository for the (longer existing) Condor DEB files. This allows you to have a decently updated Debian-based cluster with the latest version available. Great thing. Use it.

If you were using my Condor Debian packages in the past, you should stop that, and do a careful migration. The user names are equal, and the Wisconsin version of the installation scripts seems to be a little bit picky. I recommend to purge my package (for example with apt-get remove –purge condor) and check for any remainings, before you start with the new repository.

The installation is by default working in personal mode, which demands no user interaction during installation. The package looks like a completely new development, which is sad – some people (check the ITP) spend some time on things such as debconf support.

EFI support in physical and virtual machines

March 25th, 2010 by Peter

I am currently collecting the possibilities of getting EFI support in both physical and virtual machines. Here are the results I got so far. The list is constantly updated. EFI vendor and revision numbers are taken from the “ver” command in the EFI shell. Feel free to contribute !

Hardware Processor Platform EFI Vendor EFI Specification EFI Revision
Apple Computers with rEFIt 0.14 X64 Apple 1.10 4096.1
Virtualization Technology Processor Platform EFI Vendor EFI Specification EFI Revision
Virtual Box >= 3.1 IA32, X64 tbd tbd tbd
HPVM B.04.00.00 BL7 under HP-UX 11 IA64 HP 1.10 14.62

GPU in the cloud – somehow

November 4th, 2009 by Peter

HPCwire explains in a recent article how NVIDIA wants to offer access to a remote rendering cluster called ‘RealityServer’. The current description sounds more like a typical remote software offer (’software as a service’, if you prefer), and not like remotely accessible raw GPU cores, as the title suggests. Anyway, worthwhile a look…

http://www.nvidia.com/object/realityserver.html