Thursday, October 31, 2013

Happy 20th Birthday FreeBSD!

In honor of the 20th birthday of FreeBSD, we've reconstituted FreeBSD 1.0 on Qemu.  We had some help from Warren Block's  post.  Nonetheless, it was a significant challenge.

Here's how we got it up and running.  If you're up to the challenge, grab the ISO and give it a whirl.  In case you just can't get it working, I've included some links to a working Qemu image at the end of this blog post.

If you just want to copy and paste, be aware  that just copying and pasting the examples below from this website may introduce extra lines or characters.  Copy first to a local text file and clean up as needed.

First, we needed to use the stable version (0.11.1) of Qemu instead of the development version.  We had to use the Bochs bios instead of Seabios.  We could not get Seabios to recognize a second floppy drive.  Refresh your ports collection and make Qemu from source with the options shown:

# Build Qemu from ports with the options shown

cd /usr/ports/emulators/qemu
make
make install













# Set up a working directory and disk image name.
# Also setup some handy environment variables.

mkdir /usr/fbsd1_0
export QDIR=/usr/fbsd1_0
export QHDA=FreeBSD-1.0_HDA.img

# Next, grab the 1.0 ISO (cd1.iso) , mount it, copy all
# all the files to a directory we will later use for FTP.

# Something like:

mkdir -p /var/ftp/pub/FreeBSD-1.0
mdconfig -a -t vnode -f cd1.iso -u 0
mount -t cd9660 /dev/md0 /mnt
cp -av /mnt/* /var/ftp/pub/FreeBSD-1.0 

# Copy all the floppies we will need to get bootstrapped
# into our working directory:

cp -av /mnt/floppies/*.flp $QDIR


# Now create the Qemu hard disk image. We'll use the qcow
# format for now.  See the Qemu docs if you want a different
# Qemu image format.  2GB is plenty of space.


cd $QDIR
qemu-img create  -f qcow $QHDA 2G


# OK, here we go!  Light 'er up...

export QHDA=FreeBSD-1.0_HDA.img
export QDIR=/usr/fbsd1_0
/usr/local/bin/qemu -monitor stdio \
-vga cirrus \
-m 128 \
-localtime \
-fda $QDIR/kcopy_ah.flp \
-fdb $QDIR/filesyst.flp \
-hda $QDIR/$QHDA \
-boot a \
-net nic,vlan=0 \
-net user,vlan=0 \
-name "FreeBSD 1.0"

# Hit "B" to load the filesystem floppy:

















# Take the defaults except for "Install FreeBSD on entire disk" option (y)
# and MS DOS option (n), and if you want a verbose install (y).

# Do not take the default disk layout.  Give the root disk
# about 500MB (1000 sectors) and the /usr filesystem the rest.
# Everything should work (partition and newfs).
#
# Depending on the speed of your system, the disk creation
# and formatting can take 20 minutes or longer.
# Grab a cup of coffee and check your email.
# The guest installer will want you to reboot.  You can
# exit Qemu and reuse the same command line, or just hit
# return at the guest reboot prompt.

# For the 2nd boot - just hit RETURN on the disk switch prompt.
# Enter "copy" at the kc> prompt, and "wd0a" for the disk.

















# 3rd boot.  For this boot, you'll have to shut down the
# Qemu guest and restart with the cpio.flp in floppy b
# and select '1' at the prompt.  Use the command line
# below:

export QHDA=FreeBSD-1.0_HDA.img
export QDIR=/usr/fbsd1_0
/usr/local/bin/qemu -monitor stdio \
-vga cirrus \
-m 128 \
-localtime \
-fda $QDIR/kcopy_ah.flp \
-fdb $QDIR/cpio.flp \
-hda $QDIR/$QHDA \
-boot c \
-net nic,vlan=0,model=ne2k_isa \
-net tap,vlan=0,ifname=tap0 \
-name "FreeBSD 1.0"


# Files are copied from the floppy to the hard disk.
# If you selected no MS DOS option above, enter
# 'n' for "Install optional dos floppy?" prompt.
# Enter 'halt' to reboot.
















# You need networking for the next step.

# Set up for bridge/tap networking here
# and start an FTP server on the host.
# You'll need the 1.0 sources in the ftpd
# download directory you copied above (/var/ftp/pub/FreeBSD-1.0/)

# Might need to disable any firewall you have running
pfctl -d

# Start up an FTP server
/etc/rc.d/ftpd onestart

# Load some kernel modules for bridge and tap devices
kldload aio
kldload if_bridge
ifconfig bridge0 create
ifconfig bridge0

kldload if_tap
ifconfig tap0 create
ifconfig tap1 create

# Add devices to the bridge.
# Replace em0 with your network interface on the next line.
ifconfig bridge0 addm tap0 addm em0

# or, if your host is using wireless interface,

# ifconfig bridge0 addm tap0 addm wlan0

# These sysctls allow a user program (Qemu) to open the tap device.
sysctl net.link.tap.user_open=1
sysctl net.link.tap.up_on_open=1

# Make sure the bridge is actually running!
ifconfig bridge0 up


# Now boot from the hard disk.  No floppies needed.
# The OS will complain that the filesystem has
# not been checked and will run fsck to check it.
# and will then reboot by itself.


export QHDA=FreeBSD-1.0_HDA.img
export QDIR=/usr/fbsd1_0
/usr/local/bin/qemu -monitor stdio \
-vga cirrus \
-m 128 \
-localtime \
-hda $QDIR/$QHDA \
-boot c \
-net nic,vlan=0,model=ne2k_isa \
-net tap,vlan=0,ifname=tap0 \
-name "FreeBSD 1.0"

















# Qemu should now boot from hd(0a,)/386bsd
# and you should get to the shell prompt with the
# "To finish installation:"  screen.
#
# It's now time to load all the tarballs.

# We will set up a network interface (ed1)
# on the Qemu guest and FTP back to the host system.
# Both the host and the guest should be on the
# same network segment.  For this example, the
# host is at 192.168.1.200/24, and the guest is set to
# 192.168.1.150/24.  Choose the network address
# that works best for you.

# Commands indented below are entered on the GUEST, not the HOST.

      mkdir /usr/distrib
      cd /usr/distrib

      ifconfig ed1 192.168.1.150 netmask 255.255.255.0

# check the TAP interface onthe host above.
# both tap0 and bridge0 devices should be UP.
# Now FTP  into the HOST and cd to the  1.0 download area
# and get the files you need.

   ftp 192.168.1.200
      anonymous
      you@

      cd pub/FreeBSD-1.0/tarballs/bindist

      prompt off
      hash on
      binary
      mget *
      quit















 















# Extract the bindist tarballs onto the guest
# The EXTRACT.sh script produces no output by default
# and can take several minutes, possibly as long as
# 30 minutes for some tarball extractions.


      sh EXTRACT.sh


     # It looks like this:

     #!/bin/sh
     #
     # This file will extract all of the FreeBSD binaries into /.
     #
     #  SOURCEDIR=.
     #  DESTDIR=/
     #  cd $SOURCEDIR

     # Note that base.aa is REQUIRED to be able to use the source tree for
     # building in.
     #
     #  cat bin_tgz.* | gunzip | tar --directory ${DESTDIR} -xpf -

     # Wait for it to finish


     # Note that you should FTP and EXTRACT.sh the bindist,
     # objdist, srcdist, xfree86, and xfreesrc tarballs separately
     # because each has it's own EXTRACT.sh file.

     # Finally, from the install.txt doc:

     mv /.profile /.profile.install
     ln /root/.profile /.profile


     # Then read "Configuring Your System" in the install.txt guide
     # and run the command "configure"
     #
     # Now halt the guest system and shut down Qemu.
     #

# Finally, back on the host, reboot Qemu with the command:

export QHDA=FreeBSD-1.0_HDA.img
export QDIR=/usr/fbsd1_0
/usr/local/bin/qemu -monitor stdio \
-vga cirrus \
-m 128 \
-localtime \
-hda $QDIR/$QHDA \
-boot c \
-net nic,vlan=0,model=ne2k_isa \
-net tap,vlan=0,ifname=tap0 \
-name "FreeBSD 1.0"


# Hopefully, you now have a working FreeBSD 1.0 system!

# Note that, by default, there is no root password.
# Please set a root password if that is a concern for your system.

Here are some links:

Qemu image from the above, with all tarballs loaded (but no root password!):
Use the last command line above to run.

http://aorn.jimby.name:81/fbsd1.0/FreeBSD-1.0_HDA.img

MD5 (FreeBSD-1.0_HDA.img) = faf0df70ec09940e47e9a497eb113786
SHA1 (FreeBSD-1.0_HDA.img) = 2351afeaf7b86ba3542f03e257db811e36e58af6


Whimsical demo with sound by the Berlin Philharmonic. (Wait for it...)

http://aorn.jimby.name:81/fbsd1.0/fbsd_bd.mpg

MD5 (fbsd_bd.mpg) = 6565df650f0067126fa0ae51d757ee48
SHA1 (fbsd_bd.mpg) = b16044a0e2746985416e216e0b85040b3294fd74



The above demo works with VLC.  Sound under Firefox may not work.

Official FreeBSD Binary Packages now Available for pkgng

Bryan Drewery has announced the availability of the official FreeBSD pkgng repository:

We are pleased to announce that official binary packages are now available for pkg, the next generation package management tool for FreeBSD.

Pkg allows you to either use ports with portmaster/portupgrade or to have binary remote packages without ports.

We have binary packages available for i386 and amd64 on 8.3, 8.4, 9.1, 9.2, 10.0, and 11 (head).

Pkg will be the default starting in FreeBSD 10.

The pkg_install suite of tools pkg_create(1), pkg_add(1), and pkg_info(1) (which ports also use), are deprecated and will be discontinued in roughly 6 months. A communication regarding the deprecation of the pkg_install suite of tools will be sent separately in the future.

If you are currently not using pkg and wish to, run the following as root. Be sure not to add WITH_PKGNG=yes to your make.conf until after pkg is installed.

  # cd /usr/ports/ports-mgmt/pkg && make install clean
  # echo WITH_PKGNG=yes >> /etc/make.conf
  # pkg2ng

You can now either continue to use ports with portmaster/portupgrade, as before or switch to using binary packages only.

To use binary packages:
  1. Ensure your pkg(8) is up-to-date. pkg -v should say at least 1.1.4_8. If it does not, first upgrade from ports.
  2. Remove any repository-specific configuration from /usr/local/etc/pkg.conf, such as PACKAGESITE, MIRROR_TYPE, PUBKEY.
  3. If this leaves your pkg.conf empty, just remove it.
  4. mkdir -p /usr/local/etc/pkg/repos
  5. Create the file /usr/local/etc/pkg/repos/FreeBSD.conf with:
FreeBSD: {
  url: "http://pkg.FreeBSD.org/${ABI}/latest",
  mirror_type: "srv",
  enabled: "yes"
}

Note that pkg.FreeBSD.org does not have a browsable web page on it and does not have a DNS A record. This is intended as it is an SRV host. pkg(8) knows how to properly use it. You can use pkg search to browse the available packages in the repository.

Mirrors you may use instead of the global pkg.FreeBSD.org:

    pkg.eu.FreeBSD.org
    pkg.us-east.FreeBSD.org
    pkg.us-west.FreeBSD.org

Your system is now ready to use packages!

Refer to the handbook section on pkgng for usage.  Also see man pkg for examples or pkg help.


Packages are built weekly from a snapshot of the Ports Collection every Wednesday morning 01:00 UTC. They typically will be available in the repository after a few days.

Pkg 1.2 will be released in the coming month which will bring many improvements including officially signed packages. FreeBSD 10's pkg bootstrap now also supports signed pkg(8) installation.

Tuesday, October 29, 2013

FreeBSD 10.0-BETA2 Available

Glen Barber has announced the availability of BETA2. Refer to the email announcement for the image checksums. AMIs are also available for all EC2 regions.

The second BETA build of the 10.0-RELEASE release cycle is now available on the FTP servers for the amd64, i386, ia64, powerpc, powerpc64 and sparc64 architectures.

ISO images and, for architectures that support it, the memory stick images are available here or at any of the FreeBSD mirror sites.

If you notice problems you can report them through the PR system or on the -current mailing list.

If you would like to use SVN to do a source based update of an existing system, use the "stable/10" branch.

Important note to freebsd-update(8) users:  be sure to follow the instructions in the EN-13:04.freebsd-update errata notice before upgrading the system to 10.0-BETA2.
 
Changes between -BETA1 and -BETA2 include:
  • Fix AHCI ATAPI emulation when backed with /dev/cd0 in virtio.
  • Improve XHCI stability.
  • Fix 'make delete-old-libs' and 'make check-libs' to delete .debug files created by WITH_DEBUG_FILES.
  • Upgrade to the latest version of mtree from NetBSD.
  • Enable the automatic creation of a certificate for sendmail.
  • Add support for "first boot" rc.d scripts.
  • Fix jail_parallel_start="YES", ip[46].addr when interface parameter is not defined, and a bug which prevented jails from starting when $jail_conf was used and no jail name was specified.
  • Disable the Xen userland event channel driver, which is not yet ready.
  • Update T4 and T5 firmwares (cxgbe(4)).
Please note the following:

If using the ZFS installation option and full-disk encryption is enabled, a few entries will need to be manually added to loader.conf(5) before the 'bootpool' zpool will be available after the system boots.  This manual step is expected to be fixed in the next 10.0 release cycle build.

The entries that need to be added are:

        zpool_cache_load="YES"
        zpool_cache_type="/boot/zfs/zpool.cache"
        zpool_cache_name="/boot/zfs/zpool.cache"

This can be done at the final menu of bsdinstall(8), when prompted to boot into the newly-installed system; alternatively, this can be done post-install, in which case, the following must be run before appending loader.conf(5):

        # zpool import -f bootpool

Monday, October 21, 2013

FreeBSD Quarterly Status Report, July-September 2013

The latest quarterly status report is now available. From the summary:

We have had another very active three months in the FreeBSD world, including two Developer Summits (BSDCam and EuroBSDcon) that will be covered in separate status reports. FreeBSD continues to push hard on security, with improvements to both the performance and reliability of the random number generation, and more compartmentalisation in programs in the base system.

For developers, there is work on a new modern debugger. There is also a significant amount of of modernization in the support for Objective-C and Ada via ports, making FreeBSD a first-rate platform for developing in either language, in addition to the existing C++11 and C11 support already present in the base system.

Server users will be pleased to see improvements in the iSCSI stack and scalability allowing over a million I/O operations per second on commodify hardware, while desktop users will see improvements in X support for new GPUs and for possible X replacements.

Thanks to all the reporters for the excellent work! This report contains 30 entries and we hope you enjoy reading it.

Monday, October 14, 2013

FreeBSD 10.0-BETA1 now available

FreeBSD 10.0-BETA1 now available

The first beta build of the 10.0-RELEASE release cycle is now available for the amd64, i386, ia64, powerpc, powerpc64 and sparc64 architectures.  ISO images and, for architectures that support it, memory stick images are available here.

Preinstalled virtual machine images are available for the amd64 and i386 architectures, located under the 'snapshots' directory here.  The disk images are available in both QCOW2 and VMDK format.  The image download size is approximately 136 MB, which decompress to a 20GB sparse image.

The image checksums can be found in the announcement.

If you notice problems you can report them through the PR system or on the -current or -stable mailing lists.

If you would like to use SVN to do a source based update of an existing system, use the "stable/10" branch.

Important note to freebsd-update(8) users:  Due to a last minute problem found in the 10.0-BETA1 freebsd-update(8) builds, freebsd-update(8) is NOT supported for 10.0-BETA1 upgrades.  Please do not use freebsd-update(8) to upgrade to 10.0-BETA1.

Also note, due to the size of the images, the ports.txz distribution is not included in 10.0-BETA1, however is expected to be included with disc1.iso for subsequent builds during the release cycle.

The ports tree can be fetched either with the portsnap(8) utility, or using svnlite using any of the mirrors listed here.

Changes between -ALPHA5 and -BETA1 include:

  • Introduce freebsd-version(1), which is intended to be used as an auditing tool, to determine the userland patch level when it differs from what 'uname -r' reports.
  • Improve ZFS lzjb decompress performance.
  • Add two new MIPS CPU families - mips24k and mips74k.
  • The "jail_<jname>_*" rc.conf(5) variables for per-jail configuration are automatically converted to /var/run/jail.<jname>.conf before the jail(8) utility is invoked, so the new jail.conf(5) syntax is used.
  • Remove most of the ATF tools and the _atf user.
  • Updates to random(4).
    • In 10.0-BETA1, it is not possible for random(4) to be loaded as a kernel module via kldload(8).  If not using GENERIC, and the system kernel configuration excludes 'device random', please include random(4) in the kernel configuration file.  The fix for this issue is pending review, and is expected to be fixed in 10.0-BETA2.
  • Updates to bsdinstall(8).
    • 10.0-BETA1 introduces a number of updates to bsdinstall(8), notably the ability to install to a full ZFS filesystem.  Please keep in mind that this is an experimental feature.
    • If using the ZFS installation option in and have enabled full-disk encryption is enabled, a few entries will need to be manually added to loader.conf(5) before the 'bootpool' zpool will be available after the system boots.  This manual step is expected to be fixed in 10.0-BETA2.
      • The entries that need to be added are:
        • zpool_cache_load="YES"
        • zpool_cache_type="/boot/zfs/zpool.cache"
        • zpool_cache_name="/boot/zfs/zpool.cache"
      • This can be done at the final menu of bsdinstall(8), when prompted to boot into the newly-installed system; alternatively, this can be done post-install, in which case, the following must be run before appending loader.conf(5):
        • # zpool import -f bootpool


Tuesday, October 8, 2013

CFP: AsiaBSDCon 2014

Hiroki Sato has announced the dates and Call for Papers for AsiaBSDCon 2014:

I would like to announce AsiaBSDCon in the next year, 13-16 March 2014 in Tokyo.  Details are here  and the CFP can is here.

Papers, videos, and photos of the past AsiaBSDCon can also be found on the website. 2013's photos and videos are still not ready at this moment but will be added soon.  The venue of AsiaBSDCon 2014 will be the same as the past ones.

In the first two days, a DevSummit and vendor session are planned.  The details will be updated in this wiki page.
 
The paper submission deadline is November 29, 2013.  Please spread this to your friends in BSD communities and encourage them to attend (and write a paper).  Thank you!

Monday, October 7, 2013

FreeBSD 10.0-ALPHA5 Now Available

Glen Barber has announced the availability of the next ALPHA in the 10.0 series:

The fifth ALPHA build of the 10.0-RELEASE release cycle is now available for the amd64, i386, ia64, powerpc, powerpc64 and sparc64 architectures. ISO images and, for architectures that support it, memory stick images are available here.

Preinstalled virtual machine images are available for the amd64 and i386 architectures, located under the 'snapshots' directory here. The disk images are available in both QCOW2 and VMDK format.  The image download size is approximagely 136 MB, which decompress to a 10GB sparse image.

The 10.0-ALPHA5 builds correlate to svn revision r256092 of the head/ branch.

The image checksums can be found in the announcement.

If you notice problems you can report them through the PR system or on the -current mailing list.

If you would like to use SVN to do a source based update of an existing system, use the "head/" branch.

Important note to freebsd-update(8) users: freebsd-update(8) is not a supported upgrade path for the 10.0-ALPHA builds.

Changes between -ALPHA4 and -ALPHA5 include:
  • Removal of BIND from the base system; replacements may be found within the Ports Collection.
  • Bug fix introduced with the latest infiniband update.
  • The GNU ar(1) and ranlib(1) have been removed from the base system.
  • Bug fixes and updates to the base llvm.
  • Various bhyve enhancements.
  • Various XEN enhancements.