ganeti-github.git
7 years agoVersion bump to 2.8.0~alpha1
Michele Tartara [Fri, 14 Jun 2013 12:57:49 +0000 (14:57 +0200)]
Version bump to 2.8.0~alpha1

Now that alpha versions are supported, we can bump the version number
for branch stable-2.8 to 2.8.

This also requires updating the cfgupgrade tool.

Thanks to the previous patches, all the other documents can be upgraded
during the alpha lifetime, before switching to beta (that will enable
the version number checks).

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoMake cfgupgrade idempotent
Michele Tartara [Tue, 18 Jun 2013 07:33:48 +0000 (07:33 +0000)]
Make cfgupgrade idempotent

One of the unit tests checks whether cfgupgrade's downgrade option is
idempotent. It is, but a version number check made it impossible to
actually downgrade multiple times.

With this change, both the current-version version number and the
target-version version number are accepted by the downgrade tool.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoUpdate config version number when downgrading
Michele Tartara [Tue, 18 Jun 2013 07:32:31 +0000 (07:32 +0000)]
Update config version number when downgrading

The downgrade option of the cfgupgrade tool was not changing the version
number in the configuration file of the cluster while performing the
downgrade.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoAdd test for shrink heuristics over different resources
Klaus Aehlig [Fri, 14 Jun 2013 12:19:01 +0000 (14:19 +0200)]
Add test for shrink heuristics over different resources

In this example, memory prevents adding more than one instance on
the first node. Yet, on the other nodes, 2 instances can be placed
each, if disks are shrunk appropriately. This, however, requires
to courageous decision to shrink disks next, even though more nodes
fail to allocate more due to memory than to disk.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

7 years agoImprove hspace shrinking strategy
Klaus Aehlig [Fri, 14 Jun 2013 13:48:47 +0000 (15:48 +0200)]
Improve hspace shrinking strategy

In tired allocation, hspace shrinks that resource of the instance
next, that causes failure on most nodes. While, this is not a bad
strategy in general, it can lead hspace into a dead end if for a large
number of nodes a particular resource blocks any further allocation of
policy compliant instances. So we improve the heuristics in that it
chooses to shrink such a resource next where by shrinking only this
resource a valid allocation can be made, if such a resource
exists. This improves the results in some cases, while still keeping
the computational complexity of the algorithm low.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

7 years agoConvenience function for iterating while the result is Ok
Klaus Aehlig [Fri, 14 Jun 2013 15:42:15 +0000 (17:42 +0200)]
Convenience function for iterating while the result is Ok

For a function f :: a -> GenericResult a, iterate it (in the sense of the
monad), until the result is Bad; return the list of values occurred.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

7 years agoProvide witness for the sum-type structure of GenericResult
Klaus Aehlig [Fri, 14 Jun 2013 13:47:55 +0000 (15:47 +0200)]
Provide witness for the sum-type structure of GenericResult

GenericResult, while rightfully a type of its own, is isomorphic
to Either. So, also provide the case analysis function (i.e., the
universal arrow out of the sum).

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

7 years agoDisable more version checks for alpha versions
Michele Tartara [Thu, 13 Jun 2013 14:56:30 +0000 (14:56 +0000)]
Disable more version checks for alpha versions

Alpha versions should not check for version numbers in READMEs
and documentation.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoAllow alpha versions not to have an entry in the NEWS file
Michele Tartara [Thu, 13 Jun 2013 14:15:15 +0000 (14:15 +0000)]
Allow alpha versions not to have an entry in the NEWS file

Alpha versions are still under heavy development, and can therefore not have an
entry in the NEWS file yet. The entry will have to be added before switching to
the first beta version.

Partially fixes Issue 448

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoAdd tests for check-news
Michele Tartara [Thu, 13 Jun 2013 12:09:40 +0000 (14:09 +0200)]
Add tests for check-news

Unit tests verifying the behaviour of the check-news script are added.

This required adding a new make target, "autotools-check" directly called from
"commit-check", because the usual tests are executed in the build directory,
whereas these, being relative to the autotools, cannot be executed there because
the files of the autotools/ directory are not copied do not belong in the build
directory.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoIntroduce support for alpha versions
Michele Tartara [Thu, 13 Jun 2013 11:00:12 +0000 (11:00 +0000)]
Introduce support for alpha versions

An alpha version can be used to name a release while a previous one is still
being developed.

E.g. 2.9.0~alpha1 can be out while 2.8.0~rc1 is being worked on.

When a version is marked as alpha, it is allowed to have unreleased versions
before it in the NEWS file, in order to simplify merges of the NEWS file itself
between the two development branches.

Partially fixes Issue 448.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoImprove docstring
Michele Tartara [Thu, 13 Jun 2013 10:59:47 +0000 (10:59 +0000)]
Improve docstring

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoFix bug in Makefile.am
Michele Tartara [Thu, 13 Jun 2013 15:32:53 +0000 (17:32 +0200)]
Fix bug in Makefile.am

There was a typo in the name of the script for launching offline tests.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoMerge branch 'stable-2.7' into stable-2.8
Klaus Aehlig [Fri, 14 Jun 2013 08:29:40 +0000 (10:29 +0200)]
Merge branch 'stable-2.7' into stable-2.8

* stable-2.7
  Make shrinkByType aware of individual disks
  Add a test for hspace to respect instance policy

Conflicts: (trival, take union of added files/tests)
Makefile.am
test/hs/shelltests/htools-hspace.test

Semantic conflicts:
test/data/htools/hspace-tiered-ipolicy.data
(adapt to changed text format)

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

7 years agoAdd a test for hspace to respect instance policy
Klaus Aehlig [Thu, 13 Jun 2013 14:05:46 +0000 (16:05 +0200)]
Add a test for hspace to respect instance policy

In this example, the instance policy for disks prevents allocation
of more than one instance per node. The test verifies that tiered
allocation still respects this and doesn't attempt to place policy
violating instances on the cluster.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

7 years agoMake shrinkByType aware of individual disks
Klaus Aehlig [Thu, 13 Jun 2013 14:05:45 +0000 (16:05 +0200)]
Make shrinkByType aware of individual disks

When shrinking an instances, you can't just get smaller disk footprint
while leaving the individual disks as they are. Make the shrink
heuristic aware of that fact, and decrease all individual disks as
well. Fixes issue 484.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

7 years agoRestructure documentation index page
Michele Tartara [Wed, 12 Jun 2013 14:48:30 +0000 (14:48 +0000)]
Restructure documentation index page

The index page of the documentation used to be just a list of links.
Now it is a proper introduction to all the other pages.

Fixes Issue 424.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoAdd missing parenthesis to description of --machine-readable
Klaus Aehlig [Wed, 12 Jun 2013 11:45:10 +0000 (13:45 +0200)]
Add missing parenthesis to description of --machine-readable

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoUpdate NEWS file for 2.8
Michele Tartara [Tue, 14 May 2013 12:42:15 +0000 (13:42 +0100)]
Update NEWS file for 2.8

Add to the NEWS file the list of modifications performed during the 2.8
development cycle.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoAdd --force option to gnt-cluster modify
Klaus Aehlig [Tue, 4 Jun 2013 12:54:41 +0000 (14:54 +0200)]
Add --force option to gnt-cluster modify

Make the force option of the ClusterSetParams Opcode available
on the command line.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

7 years agoAdd a force option to the ClusterSetParams Opcode
Klaus Aehlig [Tue, 4 Jun 2013 12:23:13 +0000 (14:23 +0200)]
Add a force option to the ClusterSetParams Opcode

If set, the op code will, in particular, try to set the master IP
on the new netdev, even if shutting down the master IP on the old
netdev failed.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

7 years agoDocument the monitoring daemon in the admin guide
Michele Tartara [Mon, 3 Jun 2013 21:58:02 +0000 (21:58 +0000)]
Document the monitoring daemon in the admin guide

The administrator's guide must include a section describing the monitoring
daemon and its API, in order for the users to be able to use it.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoFactor out the API of the monitoring daemon
Michele Tartara [Mon, 3 Jun 2013 21:53:12 +0000 (21:53 +0000)]
Factor out the API of the monitoring daemon

It will be included by both the design document and the user documentation.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoAdd monitoring agent info to the security document
Michele Tartara [Mon, 3 Jun 2013 21:11:59 +0000 (17:11 -0400)]
Add monitoring agent info to the security document

Add information regarding the monitoring agent and its security to
the security document.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoExport device UUIDs to hooks and OS scripts
Christos Stavrakakis [Mon, 3 Jun 2013 16:31:26 +0000 (19:31 +0300)]
Export device UUIDs to hooks and OS scripts

Export UUIDs and names of instance NICs and disks to the environment
of OS scripts and instance related hooks.

Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agognt-cluster info: only parameters for enabled hypervisors
Klaus Aehlig [Mon, 3 Jun 2013 14:48:17 +0000 (16:48 +0200)]
gnt-cluster info: only parameters for enabled hypervisors

In the output of gnt-cluster info, restrict the hypervisor parameters
to those hypervisors that are enabled. This fixes issue 443.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoSupport RAPI QA on an already existing cluster
Bernardo Dal Seno [Sat, 1 Jun 2013 23:03:15 +0000 (01:03 +0200)]
Support RAPI QA on an already existing cluster

We support running most of the QA tests on an existing cluster. One big
exception was RAPI, as the credentials for the QA RAPI user were created at
cluster setup. This patch allows to recover the password for the QA user
(if configured) from an existing cluster and hence run RAPI tests.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoMerge branch 'stable-2.7' into stable-2.8
Bernardo Dal Seno [Fri, 31 May 2013 10:10:00 +0000 (12:10 +0200)]
Merge branch 'stable-2.7' into stable-2.8

* stable-2.7:
  Document users-setup tool
  Adjusting permissions after confd start
  Ensure the queue socket has the right permissions
  Update IAllocator interface documentation

Conflicts:
doc/iallocator.rst

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoDocument users-setup tool
Michele Tartara [Thu, 30 May 2013 20:42:25 +0000 (16:42 -0400)]
Document users-setup tool

The users-setup tool was added but had no documentation up to now. This commit
fixes this issue.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoAdjusting permissions after confd start
Helga Velroyen [Wed, 29 May 2013 09:40:35 +0000 (11:40 +0200)]
Adjusting permissions after confd start

This is a workaround for issue 477. Confd resets the
permissions of the query socket in a wrong way. This
patch fixes them after the start of confd.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

7 years agoEnsure the queue socket has the right permissions
Guido Trotter [Wed, 29 May 2013 05:07:30 +0000 (14:07 +0900)]
Ensure the queue socket has the right permissions

RAPI needs to be able to perform queries as well.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoUpdate IAllocator interface documentation
Thomas Thrainer [Thu, 23 May 2013 15:43:41 +0000 (17:43 +0200)]
Update IAllocator interface documentation

Update the missing documentation for the tags fields of node groups.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
(partially cherry picked from commit 2daca99b5be21ea4667637721ee2c65a14444520)

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoAdd NEWS entry for hail honoring networks
Thomas Thrainer [Thu, 23 May 2013 07:46:42 +0000 (09:46 +0200)]
Add NEWS entry for hail honoring networks

The entry also mentions the required change in the IAllocator protocol.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoAdd tests for network-aware allocation
Thomas Thrainer [Thu, 16 May 2013 12:56:26 +0000 (14:56 +0200)]
Add tests for network-aware allocation

hail-alloc-invalid-network defines a cluster with two nodegroups and an
allocation request which does not fit on any of the groups. Group 1 has
invalid disk-templates while Group 2 is not connected to the right
networks.

hail-alloc-restricted-network defines a cluster wih two nodegroups.
Nodegroup Group 1 is only used as last_resort group, but is chosen by
hail as only this group is connected to the right networks.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoHonor network connections in hail
Thomas Thrainer [Thu, 16 May 2013 14:58:00 +0000 (16:58 +0200)]
Honor network connections in hail

Before trying to allocate nodes in node groups, node groups are now
filtered based on the networks they are connected to an the networks
which are required by the new instance.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoParse NIC data from allocation request in hail
Thomas Thrainer [Tue, 21 May 2013 06:08:04 +0000 (08:08 +0200)]
Parse NIC data from allocation request in hail

Add a NIC type and extend the Instance type by a list of NIC's. Parse
the NIC's in allocation requests and store them for now. Later patches
will make use of this field in order to ensure that the requested
instance is only placed in node groups wich are connected to those
networks.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoSupport group networks in Text backend
Thomas Thrainer [Fri, 24 May 2013 08:03:02 +0000 (10:03 +0200)]
Support group networks in Text backend

The Text backend now parses network UUID (comma separated) and
serializes them in the same form.
The test data is adapted to the new format.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoParse node group networks
Thomas Thrainer [Fri, 17 May 2013 11:39:07 +0000 (13:39 +0200)]
Parse node group networks

Extend the Group by the network ids it is connected to. Adapt
the IAlloc backend such that the networks are parsed correctly.
This also required the adaption of test data.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoUpdate IAllocator interface documentation
Thomas Thrainer [Thu, 23 May 2013 07:39:33 +0000 (09:39 +0200)]
Update IAllocator interface documentation

Document the newly added network field of node groups, and also update
the missing documentation for the other fields of node groups.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoExport connected networks to IAllocator
Thomas Thrainer [Thu, 16 May 2013 11:44:10 +0000 (13:44 +0200)]
Export connected networks to IAllocator

IAllocators should not put instances on nodes in nodegroups which are
not connected to the network the instance should be connected to.
Therefore, export the networks a node group is connected to to the
IAllocator, so it can account for this restriction.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoRe-activate previously active disks in watcher
Thomas Thrainer [Wed, 15 May 2013 11:58:06 +0000 (13:58 +0200)]
Re-activate previously active disks in watcher

The watcher process previously (incorrectly) examined the admin_state
field to figure out if the disks of an instance should be activated
after a node reboot. This is changed to use the disks_active field of
instances, which always holds the correct information (even if the
instance should not be started, but only its disks activated).

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoMake the disks_active flag queryable
Thomas Thrainer [Wed, 15 May 2013 11:57:25 +0000 (13:57 +0200)]
Make the disks_active flag queryable

gnt-instance list now also supports the disks_active field.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoConsider disks_active in UpgradeConfig
Thomas Thrainer [Thu, 16 May 2013 09:13:24 +0000 (11:13 +0200)]
Consider disks_active in UpgradeConfig

The disks_active flag is updated when masterd reads the configuration.
Also, cfgupgrade now removes the disks_active flag during downgrades.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoAdd disks_active to configuration
Thomas Thrainer [Wed, 15 May 2013 11:55:30 +0000 (13:55 +0200)]
Add disks_active to configuration

This flag tracks if the disks of an instace are supposed to be active.
That's the case when an instance is running or when its disks got
activated explicitly (and in a couple of other cases).
It will be used by watcher to re-activate disks after a node reboot.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoFix release of instances in QA
Bernardo Dal Seno [Fri, 24 May 2013 23:19:07 +0000 (01:19 +0200)]
Fix release of instances in QA

Also, fix some uses of keyword arguments for arguments with a default
value, as required by the style guide, and a typo.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoMerge branch 'stable-2.7' into stable-2.8
Guido Trotter [Fri, 24 May 2013 11:19:28 +0000 (13:19 +0200)]
Merge branch 'stable-2.7' into stable-2.8

* stable-2.7:
  Bump up version for 2.7.0~rc2 release
  Add NEWS entry for SO_PEERCRED fix
  Workaround missing SO_PEERCRED
  Add debugging clause to _ExpandCheckDisks error
  Mention hail network incompatibility in manpages
  Remove obsolete Debian-related documentation
  Update NEWS for 2.7.0 rc2
  Improve installation documentation
  Fix owner of the OS log dir

Conflicts:
lib/cmdlib.py: port to cmdlib/instance_storage.py
lib/tools/ensure_dirs.py: trivial

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoBump up version for 2.7.0~rc2 release v2.7.0rc2
Guido Trotter [Fri, 24 May 2013 09:21:26 +0000 (11:21 +0200)]
Bump up version for 2.7.0~rc2 release

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoCreate overall design document for 2.8
Michele Tartara [Fri, 3 May 2013 09:11:33 +0000 (11:11 +0200)]
Create overall design document for 2.8

Also, cleanup the list of draft designs.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoAdd NEWS entry for SO_PEERCRED fix
Guido Trotter [Wed, 22 May 2013 14:55:37 +0000 (16:55 +0200)]
Add NEWS entry for SO_PEERCRED fix

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

7 years agoWorkaround missing SO_PEERCRED
Guido Trotter [Wed, 22 May 2013 12:57:51 +0000 (14:57 +0200)]
Workaround missing SO_PEERCRED

This bug happens in a few new distributions, so we workaround it by
defining the constant ourselves, if it's missing.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

7 years agoAdd debugging clause to _ExpandCheckDisks error
Guido Trotter [Wed, 22 May 2013 12:52:32 +0000 (14:52 +0200)]
Add debugging clause to _ExpandCheckDisks error

This has been reported by users, so we should have the extra debugging
available.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

7 years agoReduce pylint maximum file length to 4500
Thomas Thrainer [Wed, 22 May 2013 13:47:05 +0000 (15:47 +0200)]
Reduce pylint maximum file length to 4500

The longest Python files we still have are around 4200 lines long. In
order to prevent future growth, limit the maximum file length (checked
by pylint) to 4500 lines.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoMention hail network incompatibility in manpages
Guido Trotter [Wed, 22 May 2013 10:54:49 +0000 (12:54 +0200)]
Mention hail network incompatibility in manpages

We can't fix this in the 2.7 version, so it should be documented.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

7 years agoRemove obsolete Debian-related documentation
Thomas Thrainer [Wed, 22 May 2013 09:23:59 +0000 (11:23 +0200)]
Remove obsolete Debian-related documentation

This part of the documentation refers to Grub instead of Grub2, but
Grub2 has been the standard boot loader since Squeeze. As this part only
(wrongly) repeats the preceeding documentation, it's removed completely.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoUpdate NEWS for 2.7.0 rc2
Guido Trotter [Tue, 21 May 2013 08:00:43 +0000 (10:00 +0200)]
Update NEWS for 2.7.0 rc2

Move "local" entries at the bottom, and leave global 2.7 entries at the
top, as for the other releases.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

7 years agoImprove installation documentation
Thomas Thrainer [Tue, 21 May 2013 09:53:06 +0000 (11:53 +0200)]
Improve installation documentation

Based on user feedback the installation documentation is clarified and
extended.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>
(cherry picked from commit 3913eaa72b8a48706ca4caa59dd022fa1a05805a)

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoAdd Harep man page
Michele Tartara [Fri, 17 May 2013 15:18:46 +0000 (15:18 +0000)]
Add Harep man page

Add the man page for the Ganeti auto-repair tool.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoStash Xen config file after a failed startup
Balazs Lecz [Thu, 16 May 2013 11:32:54 +0000 (12:32 +0100)]
Stash Xen config file after a failed startup

This is a fix for a minor bug.
Currently, a failed Xen VM start results in a stale config file left
behind on the filesystem.
This change introduces a new log directory, where the Xen VM config
file is moved after a failed startup.

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoFix owner of the OS log dir
Balazs Lecz [Fri, 17 May 2013 10:23:24 +0000 (11:23 +0100)]
Fix owner of the OS log dir

The OS log dir is used by noded, not masterd.

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoMerge branch 'stable-2.7' into stable-2.8
Bernardo Dal Seno [Fri, 17 May 2013 11:12:01 +0000 (13:12 +0200)]
Merge branch 'stable-2.7' into stable-2.8

* stable-2.7:
  Update NEWS with disk creation fixes
  Don't fail to deactivate master IP if already down
  Add QA for recreating single instance disks
  Add QA for gnt-instance modify --disk
  Clean up when "gnt-instance modify" fails to create a disk
  recreate-disks honors the prealloc_wipe_disks flag
  Introduce wrapper for cmdlib._WipeDisks()
  Don't catch an exception that cannot be raised
  Wipe disks added through "gnt-instance modify"
  Support /var/run being a symlink in upload
  Final NEWS and configure.ac update for 2.7.0~rc1
  gnt-job list: deal with non-ascii encoding in jobs

Conflicts:
NEWS
lib/cmdlib.py
qa/ganeti-qa.py
qa/qa-sample.json

NEWS, qa/ganeti-qa.py and qa/qa-sample.py had trivial conflicts. But I've
updated QA changes to use the new interfaces. lib/cmdlib.py was renamed and
split, so I had to semi-manually apply the changes to the new files. I had
to change the names of some functions by removing or adding the initial
underscore and update the imported names.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

7 years agoUpdate NEWS with disk creation fixes
Bernardo Dal Seno [Fri, 17 May 2013 00:40:21 +0000 (02:40 +0200)]
Update NEWS with disk creation fixes

Also document a couple more fixes.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Thomas Thrainer <thomasth@google.com>

7 years agoSort cmdlib-related entries in Makefile.am
Thomas Thrainer [Fri, 17 May 2013 09:21:30 +0000 (11:21 +0200)]
Sort cmdlib-related entries in Makefile.am

Files in the cmdlib directory are sorted alphabetically in
Makefile.am.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agocmdlib: Cleanup public/private functions
Thomas Thrainer [Tue, 14 May 2013 12:30:08 +0000 (14:30 +0200)]
cmdlib: Cleanup public/private functions

All functions/classes which are used outside of their defining module
(with tests as an exception) no longer have a leading underscore.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agocmdlib: Extract instance query related functionality
Thomas Thrainer [Tue, 14 May 2013 12:02:29 +0000 (14:02 +0200)]
cmdlib: Extract instance query related functionality

Split instance.py further by extracting instance querying related
logical units and functions to instance_query.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agocmdlib: Extract instance operation functionality
Thomas Thrainer [Tue, 14 May 2013 11:52:28 +0000 (13:52 +0200)]
cmdlib: Extract instance operation functionality

Split instance.py further by extracting instance operations
(start/stop/reboot/etc.) related logical units and functions to
instance_operation.py.

The extracted operations have in common that they affect the operating
system in a running instance directly.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agocmdlib: Extract migration related functionality
Thomas Thrainer [Tue, 14 May 2013 11:38:23 +0000 (13:38 +0200)]
cmdlib: Extract migration related functionality

Split instance.py further by extracting migration related logical units
and functions to instance_migration.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agocmdlib: Extract storage related functionality
Thomas Thrainer [Thu, 16 May 2013 07:13:48 +0000 (09:13 +0200)]
cmdlib: Extract storage related functionality

Split instance.py further by extracting storage related logical units
and functions to instance_storage.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoReformat and define exports in cmdlib/__init__.py
Thomas Thrainer [Thu, 16 May 2013 07:12:59 +0000 (09:12 +0200)]
Reformat and define exports in cmdlib/__init__.py

cmdlib/__init__.py now simply defines the interface of the cmdlib module
by importing all classes which should be visible to clients.

Also don't ignore C0302 (Too many lines in module) any more.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoExtract miscellaneous logical units from cmdlib
Thomas Thrainer [Tue, 14 May 2013 08:24:50 +0000 (10:24 +0200)]
Extract miscellaneous logical units from cmdlib

All remaining classes in __init__.py are extracted to misc.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoExtract os related logical units from cmdlib
Thomas Thrainer [Tue, 14 May 2013 08:17:04 +0000 (10:17 +0200)]
Extract os related logical units from cmdlib

All LUOs* classes are extracted to operating_system.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoExtract query related logical units from cmdlib
Thomas Thrainer [Tue, 14 May 2013 08:07:29 +0000 (10:07 +0200)]
Extract query related logical units from cmdlib

All LUQuery* classes are extracted to query.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoExtract backup related logical units from cmdlib
Thomas Thrainer [Tue, 14 May 2013 07:55:31 +0000 (09:55 +0200)]
Extract backup related logical units from cmdlib

All LUBackup* classes are extracted to backup.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoExtract instance related logical units from cmdlib
Thomas Thrainer [Tue, 14 May 2013 07:19:16 +0000 (09:19 +0200)]
Extract instance related logical units from cmdlib

All LUInstance* classes are extracted to instance.py. Common functions
are moved to common.py if used by non-instance logical units as well.
Additionally, helper functions which are only used by LUBackup* and
LUInstance* are moved to instance_utils.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoExtract node related logical units from cmdlib
Thomas Thrainer [Mon, 13 May 2013 13:16:27 +0000 (15:16 +0200)]
Extract node related logical units from cmdlib

All LUNode* classes are extracted to node.py. Common functions are moved
to common.py if used by non-node logical units as well.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoExtract group related logial units from cmdlib
Thomas Thrainer [Mon, 13 May 2013 12:54:49 +0000 (14:54 +0200)]
Extract group related logial units from cmdlib

All LUGroup* classes are moved to group.py. Common functions are
extracted to common.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoExtract cluster related logical units from cmdlib
Thomas Thrainer [Mon, 13 May 2013 11:49:33 +0000 (13:49 +0200)]
Extract cluster related logical units from cmdlib

All LUCluster* classes are extracted to cluster.py. Shared functions are
extracted to common.py, helper functions only used by LUCluster* are
extracted to cluster.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoExtract test logical units from cmdlib
Thomas Thrainer [Mon, 13 May 2013 10:17:01 +0000 (12:17 +0200)]
Extract test logical units from cmdlib

LUTest* are moved to test.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoExtract network related logical units from cmdlib
Thomas Thrainer [Mon, 13 May 2013 09:38:08 +0000 (11:38 +0200)]
Extract network related logical units from cmdlib

LUNetwork* and associated helper functions are extracted to network.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoExtract tags related logical units from cmdlib
Thomas Thrainer [Mon, 13 May 2013 09:16:43 +0000 (11:16 +0200)]
Extract tags related logical units from cmdlib

LUTags* and their base class, TagsLU, are extracted to tags.py. An
additional shared function, _ShareAll, is extracted to common.py for
shared usage.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoExtract base classes from cmdlib
Thomas Thrainer [Mon, 13 May 2013 08:48:48 +0000 (10:48 +0200)]
Extract base classes from cmdlib

Base classes holding common functionality is extracted into base.py.
Utility functions used by both base classes and subclasses is moved to
common.py.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoDon't fail to deactivate master IP if already down
Thomas Thrainer [Tue, 14 May 2013 14:21:25 +0000 (16:21 +0200)]
Don't fail to deactivate master IP if already down

The master IP setup script now checks if the master IP is actually
configured on the machine before trying to remove the IP.

This fixes issue 460.

Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoCompatibility test for instances
Helga Velroyen [Wed, 15 May 2013 10:46:24 +0000 (12:46 +0200)]
Compatibility test for instances

This patch introduces a test to check the compatibility
of the Haskell and the Python representation of instances.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoInstance generators
Helga Velroyen [Wed, 15 May 2013 11:46:46 +0000 (13:46 +0200)]
Instance generators

This patch introduces and enhances generators for
instances:
- 'genInstWithNets' is split into the generation of an
  arbitrary instance and enhancing an instance with nets
- 'genInst' calls 'genInstWithNets' with an empty set
  of initial networks to provide a reasonable default
- the Arbitrary instance of 'Instance' uses now 'genDisks'
  to create instances with a reasonable set of disks

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoAnnotate every arbitrary instance field
Helga Velroyen [Wed, 15 May 2013 13:01:28 +0000 (15:01 +0200)]
Annotate every arbitrary instance field

The Arbitrary instance of the 'Instance' object is written
using the <*> syntax. Since it often uses the 'arbitrary'
generator for the instance's fields it is hard to figure
out which 'arbitrary' fills which instance field. This
patch annotates all fields with their name to make
maintenance of this code easier.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoGenerators for disks
Helga Velroyen [Wed, 15 May 2013 12:52:47 +0000 (14:52 +0200)]
Generators for disks

This patch adds generators for Disk instances to the Haskell
test code. It uses somewhat more reasonable generators to
fill the fields instead of just arbitrary values.
'genDiskWithChildren' is a generator that generates a disk
with a specified number of disk children. To avoid shooting
ourselves in the foot we do not generate further (grand)
child disks for the child disks. 'genDisk' calls
'genDiskWithChildren' by requesting three children as a
resonable default.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoRemove extra newline
Michele Tartara [Tue, 14 May 2013 17:26:07 +0000 (18:26 +0100)]
Remove extra newline

Also, properly set the date of the last modification.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoAdd MonD to the watcher
Michele Tartara [Mon, 13 May 2013 14:05:08 +0000 (14:05 +0000)]
Add MonD to the watcher

The monitoring daemon should always be alive, therefore it's added to the
watcher.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoStart the monitoring and node daemons together
Michele Tartara [Mon, 13 May 2013 14:00:17 +0000 (14:00 +0000)]
Start the monitoring and node daemons together

Add the monitoring daemon to the command starting the node daemon, given that
they both have to be started on all nodes.

Note that daemon-util only supports starting one daemon at the time, so the
actual command has to be composed as a sequence of two different daemon-util
invocations.

Also, the monitoring daemon invocation is conditional, depending on whether it
was enabled at configure time.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoAdd a constant stating whether monitoring is enabled
Michele Tartara [Mon, 13 May 2013 13:57:46 +0000 (13:57 +0000)]
Add a constant stating whether monitoring is enabled

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoAdd QA for recreating single instance disks
Bernardo Dal Seno [Fri, 10 May 2013 23:37:45 +0000 (01:37 +0200)]
Add QA for recreating single instance disks

So far QA only recreated the whole set of disks at once.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoAdd QA for gnt-instance modify --disk
Bernardo Dal Seno [Fri, 10 May 2013 23:23:08 +0000 (01:23 +0200)]
Add QA for gnt-instance modify --disk

Just a very basic test that adds and then removes a disk.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoClean up when "gnt-instance modify" fails to create a disk
Bernardo Dal Seno [Fri, 10 May 2013 14:27:20 +0000 (16:27 +0200)]
Clean up when "gnt-instance modify" fails to create a disk

cmdlib.LUInstanceSetParams now uses helper functions to create and wipe
disks, so that when the creation of a disk fails, any leftover device is
cleaned up. As a bonus, exceptions raised by _CreateBlockDev() are caught
correctly.

Now cmdlib._CreateDisks() is used every time there are disks to create.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agorecreate-disks honors the prealloc_wipe_disks flag
Bernardo Dal Seno [Fri, 10 May 2013 13:05:56 +0000 (15:05 +0200)]
recreate-disks honors the prealloc_wipe_disks flag

Now even recreate-disks wipes the newly-created disks, if the flag is set.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoIntroduce wrapper for cmdlib._WipeDisks()
Bernardo Dal Seno [Fri, 10 May 2013 13:44:01 +0000 (15:44 +0200)]
Introduce wrapper for cmdlib._WipeDisks()

The wrapper handles errors by logging them and cleaning up freshly-created
disks.

Also, the correct disk is used in the error message when an error happens
in cmdlib._CreateDisks() and the resulting disk clean-up fails.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoDon't catch an exception that cannot be raised
Bernardo Dal Seno [Thu, 9 May 2013 17:07:34 +0000 (19:07 +0200)]
Don't catch an exception that cannot be raised

Since 9b221ea4, _CreateBlockDev() doesn't raise OpExecError any more. Yet
some code was left in place to catch it. By removing that code we have two
advantages:
1. Dead code is removed.
2. If for whatever reason _CreateBlockDev() raises OpExecError, the
   exception is not silently dropped and we notice (so we can fix it).

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoWipe disks added through "gnt-instance modify"
Michael Hanselmann [Mon, 25 Feb 2013 15:27:27 +0000 (16:27 +0100)]
Wipe disks added through "gnt-instance modify"

In issue 353 Sascha Lucas reported that disks are not wiped when added
through “gnt-instance modify”. This patch adds this functionality and
updates the docstring for “_WipeDisks”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit 965e0e6a88e09f96d4c9b6030ab8753366c84a78)

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoMove cmdlib.py to cmdlib/__init__.py
Guido Trotter [Fri, 10 May 2013 16:12:55 +0000 (18:12 +0200)]
Move cmdlib.py to cmdlib/__init__.py

cmdlib.py has grown *really* too much. Move it into its own package to
allow splitting it further.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Thomas Thrainer <thomasth@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

7 years agoAllow build_chroot to work from any directory
Michele Tartara [Fri, 10 May 2013 13:46:51 +0000 (15:46 +0200)]
Allow build_chroot to work from any directory

build_chroot used to work only if launched from ./devel/, whereas now
it can be launched from anywhere, and it will store the resulting files
in the current directory.

Fixes Issue 459.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agobuild_chroot: check whether the data dir exists
Michele Tartara [Fri, 10 May 2013 13:45:14 +0000 (15:45 +0200)]
build_chroot: check whether the data dir exists

If the data directory is not in the expected place, the script complains
with an error message and stops, instead of giving obscure messages.

Partially fixes Issue 459.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoFix a misspelled word in design-storagetypes
Weiwei Jia [Wed, 8 May 2013 12:46:22 +0000 (20:46 +0800)]
Fix a misspelled word in design-storagetypes

Signed-off-by: Weiwei Jia <harryxiyou@gmail.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>