p2v-transfer.git
8 years agoFix unit tests to account for last two patches master
Ben Lipton [Fri, 12 Aug 2011 19:35:47 +0000 (15:35 -0400)]
Fix unit tests to account for last two patches

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoSuppress an error at the end of the transfer p2v-transfer-v0.1
Ben Lipton [Fri, 12 Aug 2011 17:15:27 +0000 (13:15 -0400)]
Suppress an error at the end of the transfer

Once the instance shuts down, the client is disconnected. So, destroy
the client so that we don't try to use it to "clean up".

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoMake sure we edit the right known_hosts file
Ben Lipton [Fri, 12 Aug 2011 16:37:31 +0000 (12:37 -0400)]
Make sure we edit the right known_hosts file

The script must be run as root, so rsync looks for known_hosts in root's
homedir. When running with sudo, ~ refers to the user's homedir instead,
so specify root's explicitly.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoIf the .ssh directory doesn't exist, create it
Ben Lipton [Fri, 12 Aug 2011 15:07:19 +0000 (11:07 -0400)]
If the .ssh directory doesn't exist, create it

Otherwise there will be an error when paramiko tries to write the
known_hosts file back

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoReorganized README like the wiki page
Ben Lipton [Thu, 11 Aug 2011 19:31:19 +0000 (15:31 -0400)]
Reorganized README like the wiki page

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoFix errors in README commands
Ben Lipton [Thu, 11 Aug 2011 19:31:18 +0000 (15:31 -0400)]
Fix errors in README commands

Also, add troubleshooting section.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoMake fix scripts device-agnostic
Ben Lipton [Fri, 12 Aug 2011 13:41:37 +0000 (09:41 -0400)]
Make fix scripts device-agnostic

Now, fix_fstab uses the same algorithm to decide what drive to use - it
takes the first one found of /dev/xvda, /dev/vda, and /dev/sda.

Also, fix a docstring.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoDynamically choose the hard disk device used in the instance
Ben Lipton [Fri, 12 Aug 2011 13:41:36 +0000 (09:41 -0400)]
Dynamically choose the hard disk device used in the instance

Checks for presence of xvda, vda, and sda, and creates the partitions on
the first one found.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoImprove handling of devices in source fstab
Ben Lipton [Wed, 10 Aug 2011 14:50:30 +0000 (10:50 -0400)]
Improve handling of devices in source fstab

Fix a couple of messages so they don't overlap, and add _GetDeviceFile,
which converts UUID= and LABEL= device specifications to the /dev/*
format required by the blockdev command. This allows handling swap
devices that are specified by UUID or label in the fstab.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAsk the user to confirm key, then insert it into known_hosts
Ben Lipton [Wed, 10 Aug 2011 14:50:29 +0000 (10:50 -0400)]
Ask the user to confirm key, then insert it into known_hosts

This should ask the user to confirm the host key, and if confirmed
insert it into the .ssh/known_hosts file so that rsync will pick it up.

Not entirely sure how the user will know if the host key is correct...

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoA few small style fixes
Iustin Pop [Tue, 9 Aug 2011 17:09:57 +0000 (19:09 +0200)]
A few small style fixes

No code changes, just some extra blank lines here and there and a
docstring change.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Benjamin Lipton <benlipton@google.com>

8 years agoComment out problematic fstab lines instead of deleting them
Ben Lipton [Tue, 9 Aug 2011 16:38:24 +0000 (12:38 -0400)]
Comment out problematic fstab lines instead of deleting them

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoOrder of filesystems in fstab is preserved
Ben Lipton [Tue, 9 Aug 2011 14:41:47 +0000 (10:41 -0400)]
Order of filesystems in fstab is preserved

Instead of using a dictionary, which iterates through its items in an
arbitrary order, return the filesystems in a list of device, mount point
pairs. Add unit tests to verify this behavior, and to check that
ParseFstab produces output in the new format.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoHandle inconsistencies in blkid
Ben Lipton [Tue, 9 Aug 2011 14:41:46 +0000 (10:41 -0400)]
Handle inconsistencies in blkid

The UUID and TYPE don't always come out of blkid in the same order,
apparently...

Current policy for which lines end up in fstab:
- / and swap partitions are modified
- noauto lines stay
- lines for "devices" that don't start with "/", (such as "proc") stay
- everything else is deleted

Note: This is a different from the current policy on which source
filesystems get mounted and transferred, but maybe it shouldn't be.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoStub out time.sleep to make unit tests a lot faster
Ben Lipton [Tue, 9 Aug 2011 14:41:45 +0000 (10:41 -0400)]
Stub out time.sleep to make unit tests a lot faster

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoBetter handling of source filesystems
Ben Lipton [Tue, 9 Aug 2011 14:41:44 +0000 (10:41 -0400)]
Better handling of source filesystems

Break out reading of source OS /etc/fstab into its own function, so we
only have to do it once. Using the results we can more reliably find the
amount of swap on the source machine. Also, we can mount and unmount all
of the disks that would normally be part of the source filesystem,
instead of being stuck with only the / partition.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoUnit tests for a couple of functions in p2v_transfer.py
Ben Lipton [Fri, 5 Aug 2011 21:20:57 +0000 (17:20 -0400)]
Unit tests for a couple of functions in p2v_transfer.py

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdded doc explaining how the bootstrap OS works
Ben Lipton [Thu, 4 Aug 2011 20:38:51 +0000 (16:38 -0400)]
Added doc explaining how the bootstrap OS works

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoFixed typos in README
Ben Lipton [Thu, 4 Aug 2011 20:38:50 +0000 (16:38 -0400)]
Fixed typos in README

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoMore useful commands in the README
Ben Lipton [Thu, 4 Aug 2011 20:38:49 +0000 (16:38 -0400)]
More useful commands in the README

Wrap commands in "gnt-cluster command" so they don't have to be run on
every node. Change ssh-keygen commands to put the keys straight into the
correct directory. Also, add a note about compatibility.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdded note to uncomment EXTRA_PKGS
Ben Lipton [Thu, 4 Aug 2011 20:38:48 +0000 (16:38 -0400)]
Added note to uncomment EXTRA_PKGS

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoWhen remote commands fail, display output locally
Ben Lipton [Wed, 3 Aug 2011 22:27:25 +0000 (18:27 -0400)]
When remote commands fail, display output locally

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd --skip-kernel-check option
Ben Lipton [Wed, 3 Aug 2011 22:27:23 +0000 (18:27 -0400)]
Add --skip-kernel-check option

For when we don't care about matching the kernel

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoNew script for upstart distros
Ben Lipton [Wed, 3 Aug 2011 22:27:22 +0000 (18:27 -0400)]
New script for upstart distros

For upstart, we create the file /etc/init/hvc0.conf instead of adding a
line to /etc/inittab.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoStill having trouble seeing error messages
Ben Lipton [Wed, 3 Aug 2011 22:27:20 +0000 (18:27 -0400)]
Still having trouble seeing error messages

Make sure we see what the error was even when we're going to retry

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoHostnames can include periods
Ben Lipton [Wed, 3 Aug 2011 22:27:19 +0000 (18:27 -0400)]
Hostnames can include periods

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoHandle missing modules
Ben Lipton [Tue, 2 Aug 2011 15:25:17 +0000 (11:25 -0400)]
Handle missing modules

Change initrd script not to crash when there is no modules dir for the
selected kernel, in case we are using a non-modular kernel. Instead,
cheat by creating an empty directory where the modules dir should be.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoCheck program args for safety
Ben Lipton [Mon, 1 Aug 2011 23:28:05 +0000 (19:28 -0400)]
Check program args for safety

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoFix weirdness in the way errors are reported
Ben Lipton [Mon, 1 Aug 2011 23:28:04 +0000 (19:28 -0400)]
Fix weirdness in the way errors are reported

Real errors were not being reported, and instead several error messages
from umount would appear. To fix this, we print out *all* exceptions
that occur, not just P2VErrors. Also, we don't try to umount when
/source is not mounted

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoRetry umount step before quitting
Ben Lipton [Mon, 1 Aug 2011 23:28:03 +0000 (19:28 -0400)]
Retry umount step before quitting

Before we give up, we should try running the umount command a couple of
times if it doesn't work, in case the filesystem was busy the first
time and is not busy anymore.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoBetter feedback for the user
Ben Lipton [Mon, 1 Aug 2011 23:28:02 +0000 (19:28 -0400)]
Better feedback for the user

p2v_transfer.py now shows the user a summary of what it is doing.
Additionally, there is now a check to see if a remote command is taking
longer than a minute. All it does is ask the user to try pinging the
machine and if it works to be patient, but it's better than no message
at all.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoChanges so that re-running the script works.
Ben Lipton [Tue, 2 Aug 2011 13:35:39 +0000 (09:35 -0400)]
Changes so that re-running the script works.

This might not be necessary, but in the event that something goes wrong
and the user wants to re-run the script, it would be nice if it didn't
fail just because /target was mounted, or simply already existed, on the
bootstrap OS. If the script fails it will still try to unmount /target,
and if the formatting step fails it will retry after sending a command
to unmount.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoSemi-intelligently guess size of swap partition
Ben Lipton [Thu, 28 Jul 2011 15:41:13 +0000 (11:41 -0400)]
Semi-intelligently guess size of swap partition

Automatically sets it to the smaller of:
* Size of the swap partition on source machine
* 10% of instance disk

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoCall run-parts to run fix scripts
Ben Lipton [Wed, 27 Jul 2011 19:21:00 +0000 (15:21 -0400)]
Call run-parts to run fix scripts

Instead of calling a nonexistent python script.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoPython 2.4 compatibility
Ben Lipton [Tue, 26 Jul 2011 21:26:13 +0000 (17:26 -0400)]
Python 2.4 compatibility

Removed references to subprocess.check_call, removed try/catch/finally
blocks, and fixed a unit test that expected behavior from
shutil.copytree that changed from 2.4 to 2.6.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoClean up temp files generated by unit tests
Ben Lipton [Tue, 26 Jul 2011 21:26:14 +0000 (17:26 -0400)]
Clean up temp files generated by unit tests

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoInstall public key, like the README says it does
Ben Lipton [Tue, 26 Jul 2011 17:14:06 +0000 (13:14 -0400)]
Install public key, like the README says it does

Also, fix the README because the directory it specifies is wrong.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd copy_hostname script
Ben Lipton [Tue, 26 Jul 2011 16:16:32 +0000 (12:16 -0400)]
Add copy_hostname script

This copies the hostname file from the bootstrap OS to the target OS,
which should presumably be the Right Thing because the p2v-target
instance will be created with its proper hostname.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd add_console_inittab script
Ben Lipton [Tue, 26 Jul 2011 16:15:29 +0000 (12:15 -0400)]
Add add_console_inittab script

If the target OS uses /etc/inittab, and the instance is running on xen,
add a line to /etc/inittab to create a console on hvc0. This doesn't
work on distros that use upstart, for example.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd remove_persistent_rules script
Ben Lipton [Tue, 26 Jul 2011 16:14:28 +0000 (12:14 -0400)]
Add remove_persistent_rules script

Removes hardware-specific udev rules

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd fix_fstab script
Ben Lipton [Tue, 26 Jul 2011 14:50:58 +0000 (10:50 -0400)]
Add fix_fstab script

Replaces filesystems in /etc/fstab with their new UUIDs.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoFix copyright headers
Ben Lipton [Tue, 26 Jul 2011 13:29:00 +0000 (09:29 -0400)]
Fix copyright headers

I was getting this wrong before, hopefully this fixes it.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoActually fix the shebang and copyright header
Iustin Pop [Tue, 26 Jul 2011 13:32:14 +0000 (15:32 +0200)]
Actually fix the shebang and copyright header

Additionally, remove empty line at EOF.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Benjamin Lipton <benlipton@google.com>

8 years agoCreate fixes directory
Ben Lipton [Mon, 25 Jul 2011 22:07:16 +0000 (18:07 -0400)]
Create fixes directory

The fixes/ directory will be recursively copied to /usr/lib/ganeti/fixes
on the bootstrap OS. It contains a few executable scripts which will be
run with run-parts. If these are to be python scripts, most of their
code should be contained in modules in fixes/fixlib/, so that they can
be unit tested.

This patch:
- Updates p2v-target.conf.in so that python will be available on the
  bootstap os
- Adds the hook script to install the fixes directory on the boostrap OS
- Creates the fixes/fixlib directory where python modules for the
  scripts will live

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
[iustin@google.com: fixed copyright headers and whitespace]
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoUpdate Makefile to include fix scripts
Ben Lipton [Mon, 25 Jul 2011 22:05:47 +0000 (18:05 -0400)]
Update Makefile to include fix scripts

instance-p2v-target/Makefile.am now installs a new dir for fix scripts
and generalizes the "building" of files that use sed to substitute
paths. Also changes the PYTHONPATH when running tests, with the effect
that scripts no longer needs to be a package for the unittests to work.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoCreated README for p2v-transfer
Ben Lipton [Thu, 21 Jul 2011 18:07:59 +0000 (14:07 -0400)]
Created README for p2v-transfer

Includes:
* Requirements for installed software
* Instructions for the admin configuring the server
* Instructions for the user making a transfer

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdded new script and tests to p2v-transfer/Makefile.am
Ben Lipton [Wed, 20 Jul 2011 18:33:43 +0000 (14:33 -0400)]
Added new script and tests to p2v-transfer/Makefile.am

Also, move dist of README, NEWS, COPYING to doc makefile so that they
are covered by normal install/uninstall in doc directory instead of the
overridden ones in root makefile, making distcheck work better.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd front-end script to perform P2V transfer.
Ben Lipton [Wed, 20 Jul 2011 18:33:42 +0000 (14:33 -0400)]
Add front-end script to perform P2V transfer.

p2v_transfer.py is run from the transfer OS to connect to the instance
and perform the transfer. It takes three arguments:
* Device that stores the root filesystem of the source OS
* Name or address of instance to connect to
* Path of a private key matching the public key installed on the
  instance.

For the time being, the administrator needs to generate the keys and
provide the private key to the user manually.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoFix building when rst2html not installed
Ben Lipton [Thu, 21 Jul 2011 14:58:40 +0000 (10:58 -0400)]
Fix building when rst2html not installed

Although it checked for rst2html, it still would fail to install if it
was missing

Signed-off-by: Ben Lipton <benlipton@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoFix build system
Ben Lipton [Thu, 14 Jul 2011 19:56:34 +0000 (15:56 -0400)]
Fix build system

Changed around a bunch of things so that it's easy to do common things
from the root directory, html documentation can be built from the .rst
files, and also handy things like 'make distcheck' work.

Signed-off-by: Ben Lipton <benlipton@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoClarified the language of the design doc.
Ben Lipton [Tue, 12 Jul 2011 20:26:40 +0000 (16:26 -0400)]
Clarified the language of the design doc.

The "Transfer Process" section now contains much more consistent
terminology, fewer references to nonexistent sections, and a shiny new
*diagram*!

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdded new files to Makefile.am
Ben Lipton [Wed, 6 Jul 2011 16:51:22 +0000 (12:51 -0400)]
Added new files to Makefile.am

Added hook scripts and put all configuration in
/etc/ganeti/instance-p2v-target

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd hook scripts to make the instance run in RAM
Ben Lipton [Tue, 12 Jul 2011 15:19:33 +0000 (11:19 -0400)]
Add hook scripts to make the instance run in RAM

I think this patch is more what I meant to do:

* Change the /etc/fstab
* Set network interfaces
* Clear root password
* Run a getty on hvc0

Also added a couple of necessary packages to EXTRA_PKGS in
p2v-target.conf.in.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoFixed build process
Ben Lipton [Fri, 1 Jul 2011 20:06:11 +0000 (16:06 -0400)]
Fixed build process

Modified Makefile.am to install the initrd build script and its unit
tests.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd make_ramboot_initrd.py and unit tests
Ben Lipton [Fri, 1 Jul 2011 20:06:10 +0000 (16:06 -0400)]
Add make_ramboot_initrd.py and unit tests

Added a script that generates an initrd suitable for running an OS
entirely from RAM, to be used as a target for physical-to-virtual
transfers.

Signed-off-by: Ben Lipton <benlipton@google.com>
[iustin@google.com: converted to epydoc formatting and other fixes]
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoFixed up .gitignore
Ben Lipton [Fri, 1 Jul 2011 20:06:09 +0000 (16:06 -0400)]
Fixed up .gitignore

.gitignore should ignore things in the correct directories, and leave
out swap and pyc files

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoFixed up configure.ac and Makefiles
Ben Lipton [Fri, 1 Jul 2011 16:44:30 +0000 (12:44 -0400)]
Fixed up configure.ac and Makefiles

Realized that the build process wasn't working because I missed some
files. The root makefile has a couple of aliases for installing the
things that belong on the source or target (node) machines.

Workflow on nodes is now:
./configure
make target

And on source machine:
./configure
make source

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdded p2v-transfer design doc
Ben Lipton [Wed, 29 Jun 2011 19:08:19 +0000 (15:08 -0400)]
Added p2v-transfer design doc

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
[iustin@google.com: re-wrapped the doc]
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoMoved instance files into subdir
Ben Lipton [Wed, 29 Jun 2011 19:08:18 +0000 (15:08 -0400)]
Moved instance files into subdir

The target instance OS definition is only part of what's needed for
p2v-transfer to work, so move it into a subdirectory to make room for
the user-side script.

Signed-off-by: Ben Lipton <benlipton@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

9 years agoUpdate NEWS file for serial console fixes v0.9
Guido Trotter [Fri, 12 Mar 2010 10:47:15 +0000 (10:47 +0000)]
Update NEWS file for serial console fixes

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

9 years agoFix serial console
Guido Trotter [Sun, 1 Nov 2009 21:23:41 +0000 (16:23 -0500)]
Fix serial console

- Binding the console on /dev/console makes it impossible to send
  control characters (eg. ctrl+C). Now that we can, on the ganeti side,
  support a better console mode, with a proper escape character, in
  order to have a working console on the instance side we explicitely
  bind getty to ttyS0, but only when the relevant hypervisor parameter
  is set (of course if that's later changed one has to do the relevant
  change inside the instance, or he'll see init error messages on the
  console)

- Ubuntu has changed to yet another style for init config. Support both.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

9 years agoRemove a couple of useless quotes
Guido Trotter [Sun, 1 Nov 2009 21:29:25 +0000 (16:29 -0500)]
Remove a couple of useless quotes

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoMake 0.9 release with the recent changes
Iustin Pop [Wed, 3 Mar 2010 09:03:31 +0000 (10:03 +0100)]
Make 0.9 release with the recent changes

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

9 years agoDistribute examples in the archive
Iustin Pop [Wed, 3 Mar 2010 08:58:24 +0000 (09:58 +0100)]
Distribute examples in the archive

The examples were not shipped at all, which makes it hard to package
them.

This patch renames the directory holding them (example → examples), adds
the executable bit on the interfaces hook, and distributes them in the
dist archive.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

9 years agoChange the customization directory
Iustin Pop [Wed, 3 Mar 2010 08:50:12 +0000 (09:50 +0100)]
Change the customization directory

The current customization directory is
“$sysconfdir/ganeti/instance-debootstrap.d”. Recently, the variants
support has added the variants configuration as
“$sysconfdir/ganeti/instance-debootstrap/variants/default.conf”. This
means we have two, very similar (instance-debootstrap.d vs.
instance-debootstrap) directories under ganeti.

This patch changes the customize directory to
$sysconfdir/ganeti/instance-debootstrap/hooks. It is more in line with
Ganeti's own hooks naming, and removes the old “.d” directory, for a
more sane layout.

People upgrading, with the default (commented-out) CUSTOMIZE_DIR will
need to rename their on-disk directory.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

9 years agoMake release 0.8 v0.8
Iustin Pop [Tue, 2 Mar 2010 10:24:51 +0000 (11:24 +0100)]
Make release 0.8

No recent changes, but we need a stable release of instance-debootstrap
for Ganeti 2.1.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

9 years agoSupport blktype as an alternative to vol_id
Guido Trotter [Mon, 19 Oct 2009 23:18:07 +0000 (19:18 -0400)]
Support blktype as an alternative to vol_id

This is needed to run when the node os is debian squeeze or ubuntu
karmik. Since blktype and vol_id need different options, we include the
options in the command variable. Moreover we add a new VOL_TYPE command,
for that usage of vol_id/blktype.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoAdd script for creating a new instance with grub
Jun Futagawa [Mon, 5 Oct 2009 17:51:55 +0000 (02:51 +0900)]
Add script for creating a new instance with grub

This example hook installs grub inside a newly created debootstrap
instance.

Signed-off-by: Jun Futagawa <jfut@integ.jp>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

9 years agoSupport API version 15 (with OS variants)
Guido Trotter [Fri, 2 Oct 2009 15:03:50 +0000 (16:03 +0100)]
Support API version 15 (with OS variants)

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

9 years agoFix two typos in a variable name
Guido Trotter [Fri, 2 Oct 2009 15:03:36 +0000 (16:03 +0100)]
Fix two typos in a variable name

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

9 years agoShip variants file with the correct name
Guido Trotter [Fri, 2 Oct 2009 15:02:51 +0000 (16:02 +0100)]
Ship variants file with the correct name

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

9 years agoInitialize the CLEANUP array and trap in common.sh
Guido Trotter [Wed, 25 Feb 2009 16:05:43 +0000 (16:05 +0000)]
Initialize the CLEANUP array and trap in common.sh

All the scripts have to inizialize this and set the cleanup trap, so
we're doing it for them in common.sh.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Olivier Tharan <olive@google.com>

10 years agoSupport os variants
Guido Trotter [Fri, 7 Aug 2009 13:01:16 +0000 (14:01 +0100)]
Support os variants

This is a Ganeti 2.1 design feature. It's not implemented yet in 2.1,
but its implementation in the debootstrap OS is totally backwards
compatible. If no OS_VARIANT environment variable is passed in,
everything proceeds as before, otherwise a instance-deboostrap tries
to load a config file from a specially configured directory, which can
be used to override any variable in the defaults file (which is sourced
at the beginning as it was before.

By default only the "default" variant is exported, which is shipped as
an empty config file.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

10 years agoMore tweaks to kpartx output parsing v0.7
Guido Trotter [Thu, 19 Feb 2009 13:42:05 +0000 (13:42 +0000)]
More tweaks to kpartx output parsing

Apparently kpartx likes to play with us, and different versions of
kpartx will give the mapped partition different names. Sometimes dashes
in the original name will be doubled, and sometimes a leading directory
in /dev (eg. a volume group name) will be included in the name itself.

In order to avoid that we just look in the output for anything which
includes the string '-1' (which is the first partition, with the
separator '-' which we forced kpartx to use with the -p- option).
Moreover, to be sure we don't pass on wrong information, we check that
our grep had some output, and that the created device actually exists
after running kpartx -a, with the output redirected to /dev/null as
otherwise sometimes it will echo a few lines which will be inserted in
the function result, and thus break the actual result filesystem_dev
variable.

Reviewed-by: iustinp

10 years agoimport: only use swap if swap_uuid is defined
Guido Trotter [Wed, 18 Feb 2009 17:05:24 +0000 (17:05 +0000)]
import: only use swap if swap_uuid is defined

As for create sometimes we'll fail in getting a uuid for the swap
disk. In that case we have to ignore that, while in import we're failing
to do so.

Reviewed-by: iustinp

10 years agoRework the blockdev handling
Iustin Pop [Wed, 18 Feb 2009 11:15:13 +0000 (11:15 +0000)]
Rework the blockdev handling

This (big) patch changes the block device handling to be the same in all
the commands. The create script is no longer generated, but instead
common.sh is.

The patch also comments out the PARTITION_STYLE entry in defaults,
otherwise we'll always use that.

Reviewed-by: ultrotter,imsnah

10 years agoUpdate NEWS file and release 0.7
Guido Trotter [Tue, 17 Feb 2009 13:51:51 +0000 (13:51 +0000)]
Update NEWS file and release 0.7

Reviewed-by: iustinp

10 years ago'msdos' is the default partition style from api10
Guido Trotter [Tue, 17 Feb 2009 13:51:38 +0000 (13:51 +0000)]
'msdos' is the default partition style from api10

But we still keep using 'none' for compatibility with ganeti 1.2 on api5

Reviewed-by: iustinp

10 years agoTo avoid bashisms, convert to /bin/bash! :)
Guido Trotter [Mon, 16 Feb 2009 10:25:44 +0000 (10:25 +0000)]
To avoid bashisms, convert to /bin/bash! :)

Reviewed-by: iustinp

10 years agoFix 1.2 compatibility
Guido Trotter [Mon, 16 Feb 2009 10:25:28 +0000 (10:25 +0000)]
Fix 1.2 compatibility

getopt must be executed outside of the get_api5_arguments function,
because there $@ will be overridden.

vol_id may return an error on old distributions (dapper) in that case we
just... err... well... ignore the error and forget to set up the swap
partition in fstab?? :)

Reviewed-by: iustinp

10 years agoAdd example instance-debootstrap.d file
Guido Trotter [Fri, 13 Feb 2009 11:18:47 +0000 (11:18 +0000)]
Add example instance-debootstrap.d file

This configures the eth0 interface for dhcp.

Reviewed-by: iustinp

10 years agoAdd CUSTOMIZE_DIR to personalize the installation
Guido Trotter [Fri, 13 Feb 2009 11:18:29 +0000 (11:18 +0000)]
Add CUSTOMIZE_DIR to personalize the installation

The scripts in this directory, if defined, will be executed using
run-parts, allowing some customization of the instance after the
debootstrap OS has acted and before handing back the instance to ganeti.

Reviewed-by: iustinp

10 years agoImplement PARTITION_STYLE='msdos'
Guido Trotter [Wed, 28 Jan 2009 12:03:34 +0000 (12:03 +0000)]
Implement PARTITION_STYLE='msdos'

With this partition style the blockdev passed in from ganeti is used as
a whole disk, a partition table is written on it, then a single primary
partition spanning the whole disk, and at last the os is installed in
this partition.

Reviewed-by: iustinp

10 years agoAdd the PARTITION_STYLE configuration variable
Guido Trotter [Wed, 28 Jan 2009 12:03:16 +0000 (12:03 +0000)]
Add the PARTITION_STYLE configuration variable

This variable will be used to decide if and how to partition the
instance block device.

Reviewed-by: iustinp

10 years agoinstance-debootstrap: release 0.6 v0.6
Guido Trotter [Mon, 26 Jan 2009 15:58:39 +0000 (15:58 +0000)]
instance-debootstrap: release 0.6

This commit also updates the NEWS file

Reviewed-by: iustinp

10 years agodevel/upload: add a missing wait
Guido Trotter [Mon, 26 Jan 2009 15:58:26 +0000 (15:58 +0000)]
devel/upload: add a missing wait

Without this the script exits, and removes the temporary directory,
before rsync is done.

Reviewed-by: iustinp

10 years agoUse losetup rather than -o loop
Guido Trotter [Mon, 26 Jan 2009 12:19:31 +0000 (12:19 +0000)]
Use losetup rather than -o loop

If the device we're given is actually a file, rather than just mounting
it with the -o loop option, we'll explicitely losetup it to a device,
and losetup -d (detach) it at the end. This allows us to do more actions
on the target device (for example partition it, if needed).

Reviewed-by: iustinp

10 years agoAbstract cleanups in a CLEANUP array
Guido Trotter [Mon, 26 Jan 2009 12:19:19 +0000 (12:19 +0000)]
Abstract cleanups in a CLEANUP array

Rather than just having two cleanups for the whole script (namely the
umounting and removal of the temp directory) we can change cleanups with
an array structure, which is then executed in reverse order. This will
be used to add more operations that need their own cleanup, in a
structured way.

Reviewed-by: iustinp

10 years agoAdd devel/upload script
Guido Trotter [Mon, 26 Jan 2009 10:34:30 +0000 (10:34 +0000)]
Add devel/upload script

This eases development/debugging of the os script.
It's mostly copied from its ganeti counterpart.

Reviewed-by: iustinp

10 years agodefaults: update EXTRA_PKGS comments
Guido Trotter [Mon, 26 Jan 2009 10:34:18 +0000 (10:34 +0000)]
defaults: update EXTRA_PKGS comments

EXTRA_PKGS comments used to be xen specific, and also a bit outdated.
With this patch we make them hypervisor agnostic, and then we provide
some sane defaults for different hypervisors/distro. Moreover we no
longer state that lenny doesn't have xen support for amd64, as it does.

Reviewed-by: iustinp

10 years agoinstance debootstrap: enable import
Guido Trotter [Wed, 22 Oct 2008 14:09:42 +0000 (14:09 +0000)]
instance debootstrap: enable import

Reviewed-by: iustinp

10 years agoinstance debootstrap: enable export
Guido Trotter [Wed, 22 Oct 2008 14:09:28 +0000 (14:09 +0000)]
instance debootstrap: enable export

Avoid exporting swap devices

Reviewed-by: iustinp

10 years agocommon.sh: output all errors to stderr
Guido Trotter [Wed, 22 Oct 2008 14:09:16 +0000 (14:09 +0000)]
common.sh: output all errors to stderr

Create a log_error function that outputs to stderr and use it to send
error messages. This also clarifies error messages a bit and fixes one
wrong check for new_name rather than old_name.

Reviewed-by: iustinp

10 years agoUse basename when checking the script name
Guido Trotter [Wed, 22 Oct 2008 14:09:04 +0000 (14:09 +0000)]
Use basename when checking the script name

Also save it to the SCRIPT_NAME variable to avoid calling basename every
time or forgetting to call it.

Reviewed-by: iustinp

10 years agosearch for vol_id and use retrocompatible options
Guido Trotter [Wed, 22 Oct 2008 14:08:52 +0000 (14:08 +0000)]
search for vol_id and use retrocompatible options

older versions of udev, for example the one on dapper, install vol_id in
/lib/udev but not in /sbin. They also require shorter options, which
happen to work on newer udev as well.

Reviewed-by: iustinp

10 years agouse proper variable name INSTANCE_NAME
Guido Trotter [Fri, 17 Oct 2008 14:21:41 +0000 (14:21 +0000)]
use proper variable name INSTANCE_NAME

the design doc the api version 10 uses the INSTANCE_NAME variable name
to refer to an instance, but we were incorrectly expecting INSTANCE

Reviwed-by: iustinp

10 years agoShip common.sh with the debootstrap os
Guido Trotter [Fri, 17 Oct 2008 14:21:30 +0000 (14:21 +0000)]
Ship common.sh with the debootstrap os

Reviwed-by: iustinp

10 years agoOLD/NEW swap, fix get_api10_arguments too
Guido Trotter [Thu, 16 Oct 2008 16:38:53 +0000 (16:38 +0000)]
OLD/NEW swap, fix get_api10_arguments too

Reviewed-by: iustinp

10 years agorename: invert old and new instance name
Guido Trotter [Thu, 16 Oct 2008 13:35:31 +0000 (13:35 +0000)]
rename: invert old and new instance name

For compatibility with what ganeti internally does the name passed in
INSTANCE_NAME is the new one, and the old one is passed in
OLD_INSTANCE_NAME.

Reviewed-by: iustinp

10 years agoinstance-debootstrap: Support both api 5 and 10
Guido Trotter [Thu, 16 Oct 2008 09:20:36 +0000 (09:20 +0000)]
instance-debootstrap: Support both api 5 and 10

Reviewed-by: iustinp