ganeti-github.git
7 years agoSet the default editor inside the chroot
Michele Tartara [Wed, 10 Apr 2013 08:48:41 +0000 (08:48 +0000)]
Set the default editor inside the chroot

Allows the chroot builder to set the default editor to be used in the
chroot.

Vim is used as the default choice.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoAdd backports to the chroot builder sources
Michele Tartara [Tue, 9 Apr 2013 15:28:08 +0000 (15:28 +0000)]
Add backports to the chroot builder sources

This allows to have newer software than what is available in Debian Squeeze
(the default version of the distribution used by the script), particularly
vim and git.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoqa: Fix mistake in _DestroyInstanceDisks
Guido Trotter [Mon, 8 Apr 2013 22:23:35 +0000 (18:23 -0400)]
qa: Fix mistake in _DestroyInstanceDisks

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

7 years agoqa: fix instance recreate disks for ST_FILE disks
Guido Trotter [Mon, 8 Apr 2013 07:43:54 +0000 (09:43 +0200)]
qa: fix instance recreate disks for ST_FILE disks

More lvm commands "sprinkled" in qa :/ We really have to factor them out
in the storage layer, once we have a good one.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

7 years agoqa: disable inter-cluster-instance-move for file
Guido Trotter [Sun, 7 Apr 2013 05:53:37 +0000 (07:53 +0200)]
qa: disable inter-cluster-instance-move for file

This actually hides a real issue that was discovered by enabling
file-based qa, so bug was filed for it. On the other hand it's better
than keeping the qa broken, since this functionality was broken anyway.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

7 years agoqa: fix RemoveInstanceOfflineNode for file-based
Guido Trotter [Sun, 7 Apr 2013 02:23:58 +0000 (04:23 +0200)]
qa: fix RemoveInstanceOfflineNode for file-based

- The cleanup should only happen for non externally mirrored disk
  templates
- It should behave differently for file and lvm based instances
- It should definitely *not* be explicit in the qa file :/ (but it's
  more important to fix qa, as of now)

Also fix a typo in the same function's docstring.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

7 years agoqa: skip gnt-backup for file instances
Guido Trotter [Sun, 7 Apr 2013 02:22:34 +0000 (04:22 +0200)]
qa: skip gnt-backup for file instances

Note that this fixes the "current issue" but doesn't fix the underlying
problem. :/

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

7 years agoqa: pass some extra options to make scp work
Guido Trotter [Sat, 6 Apr 2013 19:33:15 +0000 (21:33 +0200)]
qa: pass some extra options to make scp work

scp by itself won't work on a cluster: a few extra options are needed to
tell it where to find the "right" ssh global known hosts file managed by
Ganeti.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoqa: fix modify primary test
Guido Trotter [Sat, 6 Apr 2013 18:45:20 +0000 (20:45 +0200)]
qa: fix modify primary test

This test uses a non-existing config value "file-storage-dir" and fails.
Fixing it by using the default value, which is currently what qa does.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoqa: fix rename instance for non-lvm backed storage
Guido Trotter [Sat, 6 Apr 2013 16:35:34 +0000 (18:35 +0200)]
qa: fix rename instance for non-lvm backed storage

The rename instance test performs an lvm-specific check for tags on the
instance volume. This should be in the future abstracted *inside* the
template class, and not explicitly called from qa.

This would allow other disks to perform the checks that make sense for
them. In the meantime though, we just make the same check only when the
backend storage for the instance is lvm-based.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoCheck for python-yaml at configure time
Bernardo Dal Seno [Fri, 5 Apr 2013 16:13:15 +0000 (18:13 +0200)]
Check for python-yaml at configure time

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

7 years agoAdd script for building chroot
Michele Tartara [Tue, 2 Apr 2013 13:03:49 +0000 (15:03 +0200)]
Add script for building chroot

Setting up a proper environment for building a Ganeti can be quite difficult.

This patch adds a script for building a compressed chroot with all the
required libraries and tools.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoExtension of storage reporting design doc
Helga Velroyen [Mon, 25 Mar 2013 14:17:17 +0000 (15:17 +0100)]
Extension of storage reporting design doc

This patch rewrites and extends the design doc about storage reporting
with respect to disk templates and storage types. In constrast to the
previous version, we now consider disk templates as the user-facing
entity, that the user can dis/enable for the cluster. Storage types
on the other hand describe the underlying technology used by the various
disk templates. Storage reporting will use a mapping from disk templates
to storage types to pick the correct method to report the storage for
the respective disk templates.

Note that the design doc in this state still contains some questions and
FIXMEs. Feel free to comment on those. I will complete them directly or
in future patches.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoRemove unused code
Bernardo Dal Seno [Wed, 3 Apr 2013 11:55:54 +0000 (13:55 +0200)]
Remove unused code

After changing the way gnt-xxx info print their output,
cli.FormatParameterDict() is not used anymore.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agognt-node info uses a revised format
Bernardo Dal Seno [Wed, 3 Apr 2013 11:50:26 +0000 (13:50 +0200)]
gnt-node info uses a revised format

The code is more modular, and the output is YAML-compliant. QA has been
updated to reflect the new format.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agognt-group info uses a revised format
Bernardo Dal Seno [Thu, 28 Mar 2013 11:45:06 +0000 (12:45 +0100)]
gnt-group info uses a revised format

The code is more modular, the output is more similar to "gnt-cluster info"
and is YAML-compliant.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoRefactor out function to format instance policy
Bernardo Dal Seno [Thu, 28 Mar 2013 11:34:31 +0000 (12:34 +0100)]
Refactor out function to format instance policy

The new function can be used for group-level instance policies. Support for
roman numeral formatting has been dropped to make the code simpler. For
other info items it's already unsupported, and it's not exactly documented
anyway.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoFix help string for gnt-group info
Bernardo Dal Seno [Tue, 2 Apr 2013 20:32:35 +0000 (22:32 +0200)]
Fix help string for gnt-group info

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoDocument procedure to downgrade a cluster
Bernardo Dal Seno [Wed, 3 Apr 2013 15:38:58 +0000 (17:38 +0200)]
Document procedure to downgrade a cluster

This is based on the --donwgrade option of cfgupgrade, first introduced in
1709435e.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoFix upgrade procedure in UPGRADE
Bernardo Dal Seno [Wed, 3 Apr 2013 15:05:35 +0000 (17:05 +0200)]
Fix upgrade procedure in UPGRADE

The watcher should be disabled while upgrading.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoFix error in the developer notes
Michele Tartara [Thu, 28 Mar 2013 09:19:38 +0000 (09:19 +0000)]
Fix error in the developer notes

The version numbers of two programs had been swapped.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoAdd dependency on PyYAML in devnotes
Bernardo Dal Seno [Tue, 2 Apr 2013 12:01:21 +0000 (14:01 +0200)]
Add dependency on PyYAML in devnotes

This is required because of 0e79564a; otherwise, the pylint-qa target will
fail.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoAdd QA for gnt-instance modify --new-primary
Klaus Aehlig [Thu, 14 Mar 2013 11:52:49 +0000 (12:52 +0100)]
Add QA for gnt-instance modify --new-primary

Add a simple walk through the intended use case of
gnt-instance modify --new-primary for the disk template
file. The tested scenario is shutting down an instance,
manually moving the disk to the new node, setting the
new primary, and starting up the instance on the new
node. It is also verified that the operation is rejected
while the instance is still running on the old node.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoAdditionally support disk-type file in QA
Klaus Aehlig [Thu, 14 Mar 2013 11:52:48 +0000 (12:52 +0100)]
Additionally support disk-type file in QA

Also perform QA tests for the file disk template. The main reason
is that testing gnt-instance modify --new-primary requires a
manual move of disks, which is most easily simulated for files.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoAdd option --new-primary to gnt-instance modify
Klaus Aehlig [Thu, 14 Mar 2013 11:52:48 +0000 (12:52 +0100)]
Add option --new-primary to gnt-instance modify

Extend gnt-instance modify by a new parameter --new-primary
that sets the pnode paramter of the opcode.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoMake LUInstanceSetParams handle pnode parameter
Klaus Aehlig [Thu, 14 Mar 2013 11:52:47 +0000 (12:52 +0100)]
Make LUInstanceSetParams handle pnode parameter

If pnode is set LUInstanceSetParams now sets the primary node
accordingly. Unless force is set, it is verified that the instance
is no longer running on the old node.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoAdd parameter pnode to OpInstanceSetParams
Klaus Aehlig [Thu, 14 Mar 2013 11:52:47 +0000 (12:52 +0100)]
Add parameter pnode to OpInstanceSetParams

This parameter will be used to set the new primary node
of an instance, assuming that the disks have been moved
by other means (outside ganeti's control).

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoUpdate installation documentation
Michele Tartara [Tue, 2 Apr 2013 08:29:02 +0000 (10:29 +0200)]
Update installation documentation

Some required dependencies were missing from the installation instructions.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoCorrect typos in comments
Michele Tartara [Fri, 15 Mar 2013 10:08:15 +0000 (10:08 +0000)]
Correct typos in comments

Correct four typos in the comments of the Drbd types file.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoAdd status information to the DRBD data collector report
Michele Tartara [Mon, 28 Jan 2013 17:13:43 +0000 (17:13 +0000)]
Add status information to the DRBD data collector report

Add status information as required by the design document.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoExport the data collected by DRBD in JSON format
Michele Tartara [Thu, 14 Mar 2013 13:34:36 +0000 (13:34 +0000)]
Export the data collected by DRBD in JSON format

Write an intermediate function computing the JSON data before adding them
to the DCReport.

Directly export the data computed by that function (with default parameters)
allowing them to be used (in the future) by the monitoring agent.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoAdd DCStatus data type for the data collectors
Michele Tartara [Thu, 14 Mar 2013 13:09:27 +0000 (13:09 +0000)]
Add DCStatus data type for the data collectors

Also adds the DCStatusCode, part of DCStatus, and the addStatus
utility function for adding the "status" field to an already existing
JSValue.

The design document is updated to have the status codes sorted by increasing
seriousness.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoAdd Kind data type for data collectors
Michele Tartara [Thu, 14 Mar 2013 12:47:24 +0000 (12:47 +0000)]
Add Kind data type for data collectors

Also, add it to the DRBD data collector, and export it from there.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoAdd data type for data collector category
Michele Tartara [Thu, 14 Mar 2013 12:39:23 +0000 (12:39 +0000)]
Add data type for data collector category

Also, update the DRBD data collector to use and export it.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoExport the dcVersionInformation for the Drbd collector
Michele Tartara [Thu, 14 Mar 2013 12:02:26 +0000 (12:02 +0000)]
Export the dcVersionInformation for the Drbd collector

Also, update the JSON output (and the design document) so that it is not
in camelcase anymore. This is part of a bigger effort to remove camelcase
from the exposed JSON.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoAdd data collector version data type
Michele Tartara [Thu, 14 Mar 2013 11:50:41 +0000 (11:50 +0000)]
Add data collector version data type

Define the new data type and update the DRBD data collector to use it.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoFix comment line position in DRBD data collector
Michele Tartara [Thu, 14 Mar 2013 13:41:43 +0000 (13:41 +0000)]
Fix comment line position in DRBD data collector

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoExport dcName information of Drbd data collector
Michele Tartara [Thu, 14 Mar 2013 10:40:52 +0000 (10:40 +0000)]
Export dcName information of Drbd data collector

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoChange format of the empty status message
Michele Tartara [Thu, 7 Mar 2013 17:36:42 +0000 (18:36 +0100)]
Change format of the empty status message

Having 'null' instead of an empty string for an empty status message
increases the complexity of the code with no real improvement.

This patch modifies the monitoring agent design document solving this issue.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

7 years agoImprove INSTALL documentation
Michele Tartara [Thu, 28 Mar 2013 12:43:50 +0000 (12:43 +0000)]
Improve INSTALL documentation

Additional clarification note added.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoDocument more installation dependencies
Michele Tartara [Thu, 28 Mar 2013 10:10:35 +0000 (10:10 +0000)]
Document more installation dependencies

A couple of them were missing and are added by this patch.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoUnit tests for config.ConfigWriter.VerifyConfig()
Bernardo Dal Seno [Wed, 20 Feb 2013 15:14:31 +0000 (16:14 +0100)]
Unit tests for config.ConfigWriter.VerifyConfig()

The test is aimed mostly at instance policies, which changed recently.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agocfgupgrade: Add support for newly refactored ipolicies
Bernardo Dal Seno [Tue, 26 Mar 2013 22:22:06 +0000 (23:22 +0100)]
cfgupgrade: Add support for newly refactored ipolicies

Instance policies in the configuration are upgraded and downgraded. Both
operations are idempotent.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoRefactor ispecs in ipolicy structures
Bernardo Dal Seno [Tue, 19 Feb 2013 21:14:13 +0000 (22:14 +0100)]
Refactor ispecs in ipolicy structures

Minimum and maximum instance specs are put together into a single element
of the instance policy. This is in preparation for introducing multiple
min/max specs.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agocfgupgrade: Add --downgrade option
Bernardo Dal Seno [Wed, 27 Feb 2013 14:28:12 +0000 (15:28 +0100)]
cfgupgrade: Add --downgrade option

It's now possible to go back to the previous stable version. Unit tests
provided.

This is mostly useful during development, when going from master to
stable/devel.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agocfgupgrade: Refactor code
Bernardo Dal Seno [Wed, 27 Feb 2013 14:21:59 +0000 (15:21 +0100)]
cfgupgrade: Refactor code

All the upgrading code is now in one function. No functionality has been
changed.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoConfiguration is verified on load
Bernardo Dal Seno [Wed, 27 Feb 2013 18:05:21 +0000 (19:05 +0100)]
Configuration is verified on load

If there are errors in the configuration when master daemon is started,
they are now logged and reported. Previously they were logged sometimes at
startup, and when performing any change to the configuration.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoRefactor cli.CreateIPolicyFromOpts()
Bernardo Dal Seno [Wed, 13 Mar 2013 20:46:07 +0000 (21:46 +0100)]
Refactor cli.CreateIPolicyFromOpts()

The code relative to instance specs has been moved into a different
function. This in preparation for further patches to have a more clear code
structure.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoRemove duplicate filling of instance policy
Bernardo Dal Seno [Wed, 6 Mar 2013 23:19:19 +0000 (00:19 +0100)]
Remove duplicate filling of instance policy

Instance policy fields were manually filled in CreateIPolicyFromOpts() and
then filled again by the caller. Now they are filled once inside the
function.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoUnit test for cli.CreateIPolicyFromOpts()
Bernardo Dal Seno [Thu, 14 Mar 2013 16:46:57 +0000 (17:46 +0100)]
Unit test for cli.CreateIPolicyFromOpts()

Useful for subsequent patches.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoAmend partitioned design doc for multiple ispecs
Bernardo Dal Seno [Wed, 20 Mar 2013 00:48:07 +0000 (01:48 +0100)]
Amend partitioned design doc for multiple ispecs

There will be only one standard specification in instance policies.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoQA: Fix assertion
Bernardo Dal Seno [Wed, 27 Mar 2013 10:22:49 +0000 (11:22 +0100)]
QA: Fix assertion

The assertion was always true because "all" was missing.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

7 years agoMore info on using the RAPI
Michele Tartara [Tue, 26 Mar 2013 16:21:43 +0000 (16:21 +0000)]
More info on using the RAPI

Add some additional information about how to use/test the RAPI,
including how to specify authentication credentials.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agognt-instance info uses a revised format
Bernardo Dal Seno [Wed, 13 Mar 2013 17:22:58 +0000 (18:22 +0100)]
gnt-instance info uses a revised format

The code is more modular and the output is YAML-compliant. QA updated to
take advantage of it.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agognt-cluster info uses a revised format
Bernardo Dal Seno [Mon, 25 Mar 2013 00:58:16 +0000 (01:58 +0100)]
gnt-cluster info uses a revised format

The code is more modular and the output is YAML-compliant.

Added function in QA that uses PyYAML to parse the command output, and QA
is updated to take advantage of it (instead of using lots of complicated
REs).

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoGeneric function to print data for gnt-xxx info
Bernardo Dal Seno [Wed, 13 Mar 2013 17:22:50 +0000 (18:22 +0100)]
Generic function to print data for gnt-xxx info

The function produces a human-readable output, which is also a valid YAML
file, from an intermediate data structure. This will be used by "gnt-xxx
info" commands. The output of "gnt-xxx info" was almost YAML-compliant, and
only minor adjustments are needed to make it fully compliant.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoFix bug in rlib2 unit tests
Michele Tartara [Thu, 21 Mar 2013 15:54:43 +0000 (16:54 +0100)]
Fix bug in rlib2 unit tests

The "queryargs" parameter of the __init__ function of rlib2 classes should
be a dictionary, as defined in the ResourceBase parent class (in
lib/rapi/baserlib.py).

In the rlib2 and baserlib unit tests, when the queryargs parameter is not
actually needed, it is sometimes erroneously initialized with an empty list or a
"None" value instead of an empty dictionary.

This commit solves the problem and introduces an assertion to prevent it from
happening again in the future.
The use of the assertion is safe, because in production code the __init__
function is only called by the RAPI server, with the queryargs parameter
initialized by the Mapper.getController function (lib/rapi/connector.py) that
always returns a dictionary. It can only affect test code and future code.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoRemove use of deprecated function
Michele Tartara [Fri, 22 Mar 2013 11:37:18 +0000 (11:37 +0000)]
Remove use of deprecated function

cgi.parse_qs() is deprecated in python 2.6 and substituted by
urlparse.parse_qs().

This patch modifies the code to use the new function.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agognt-cluster modify: limit storage type consistency check
Helga Velroyen [Fri, 22 Mar 2013 10:31:46 +0000 (11:31 +0100)]
gnt-cluster modify: limit storage type consistency check

This fixes the currently broken Q&A. The consistency check
for storage types is now only run when the 'enabled_storage_types'
are actually changed by 'gnt-cluster modify'.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoCheck if instances use disk templates that get disabled
Helga Velroyen [Thu, 21 Mar 2013 12:06:01 +0000 (13:06 +0100)]
Check if instances use disk templates that get disabled

When disk templates get disabled via 'gnt-cluster modify', so far
we did not check if any instances still use a disk template of that
type. This patch adds the necessariy check.
It also includes adding a map of disk templates to storage types
and reordering of storage types and disk templates in alphabetical order.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoFixing some mistakes in design doc for space reporting
Helga Velroyen [Wed, 20 Mar 2013 14:25:33 +0000 (15:25 +0100)]
Fixing some mistakes in design doc for space reporting

- Use entities as examples for storage types that are actually storage
types.
- Fix mixing up 'external_storage' and 'exclusive_storage'.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoUse 'storage type' consistently in design doc
Helga Velroyen [Wed, 20 Mar 2013 10:10:28 +0000 (11:10 +0100)]
Use 'storage type' consistently in design doc

To minimize the confusion between whether or not a storage type
is the same as a storage method, I updated the design doc to
use 'storage type' instead of 'storage method' consistently.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoUpgrade configuration wrt enabled storage types
Helga Velroyen [Wed, 20 Mar 2013 13:51:23 +0000 (14:51 +0100)]
Upgrade configuration wrt enabled storage types

This implements an online update of the configuration for when a configuration
is loaded that does not yet contain the 'enabled_storage_types' attribute.
Note that this will require more changes as dis/enabling of (shared) file
storage at configure time will be removed.
This patch includes adding a mapping from disk templates to storage types. While
I was on it, I also reordered the disk template and storage type parameters
alphabetically, to make dealing with those easier.
Also, this mentions the configuration change in the NEWS file.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoQA: Extend cluster QA wrt enabled storage types
Helga Velroyen [Thu, 14 Mar 2013 14:09:35 +0000 (15:09 +0100)]
QA: Extend cluster QA wrt enabled storage types

This extends the QA scripts to test the changes in 'gnt-cluster init',
'gnt-cluster info', and 'gnt-cluster modify' with respect to enabled
storage types.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agognt-cluster init: add enabled_storage_types parameter
Helga Velroyen [Thu, 14 Mar 2013 10:49:21 +0000 (11:49 +0100)]
gnt-cluster init: add enabled_storage_types parameter

This patch extends the 'gnt-cluster init' command to set the list
of enabled storage types on cluster creation.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agognt-cluster modify: dis/enabling storage types
Helga Velroyen [Wed, 13 Mar 2013 15:28:45 +0000 (16:28 +0100)]
gnt-cluster modify: dis/enabling storage types

This patch extends the 'gnt-cluster modify' command to manipulate the list
of enabled storage types. Note that this currenlty does no validation
with respect to whether or not there are instances currently using a storage
type that is being removed from the list.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agognt-cluster info: show enabled storage types
Helga Velroyen [Wed, 13 Mar 2013 14:38:17 +0000 (15:38 +0100)]
gnt-cluster info: show enabled storage types

This extends the 'gnt-cluster info' command to list the storage types
that are enabled on the cluster. It also fixes the broken indentation
in the 'handleCall' function.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoAdd 'enabled_storage_types' to the cluster config
Helga Velroyen [Wed, 13 Mar 2013 13:44:20 +0000 (14:44 +0100)]
Add 'enabled_storage_types' to the cluster config

This patch adds the cluster's new field 'enabled_storage_types'
to the configuration objects in python and haskell.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoAdd constants for storage types to constants.py
Helga Velroyen [Wed, 13 Mar 2013 12:21:41 +0000 (13:21 +0100)]
Add constants for storage types to constants.py

This patch adds the currently known and used storage methods to the
constants.py file. Note that storage types are different from disk
templates. For example both disk templates, drbd and plain, use lvm
storage types.

The rapi documentation asserts that valid storage types are limited
to lvm and file. With the extension of storage types, this is not true
anymore, but the implementation of listing storage units is not
implemented for the new types. We disable the assert for now, adding
a fixme to extend it once the listing of storage units is extended
for the new types as well.

'gnt-node list-storage' currently does not support any other storage
type than 'lvm' and 'file'. We limit the QA to storage types that are
currently implemented for this operation.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoFix Mond man page
Michele Tartara [Thu, 21 Mar 2013 11:18:26 +0000 (12:18 +0100)]
Fix Mond man page

Remove some erroneous formatting from the Mond man page.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoAdd monitoring HTTP API structure
Michele Tartara [Thu, 7 Mar 2013 15:36:48 +0000 (15:36 +0000)]
Add monitoring HTTP API structure

Add all the supported commands to the API.
The actual response is still to be implemented.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoAdd basic HTTP server functionalities to Mond
Michele Tartara [Fri, 8 Mar 2013 18:27:06 +0000 (18:27 +0000)]
Add basic HTTP server functionalities to Mond

Add a stub implementation of the Mond HTTP server to Mond using the Haskell
snap-server library.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

7 years agoUpdate documentation for monitoring daemon
Michele Tartara [Fri, 8 Mar 2013 18:35:41 +0000 (19:35 +0100)]
Update documentation for monitoring daemon

This commit updates the installation instructions and the user guide.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

7 years agoMerge branch 'devel-2.7'
Bernardo Dal Seno [Thu, 21 Mar 2013 09:42:16 +0000 (10:42 +0100)]
Merge branch 'devel-2.7'

* devel-2.7
  cfgupgrade: Fix error messages
  Update "FIXME" string in RAPI
  rapi client: add target_node to migrate instance
  Make diskless instances externally mirrored
  Fix migrate/failover -n for ext mirror storage

Trivial merge: no conflicts

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoAdd the core of the monitoring daemon
Michele Tartara [Wed, 20 Feb 2013 10:39:17 +0000 (11:39 +0100)]
Add the core of the monitoring daemon

This commit adds the core infrastructure of the monitoring daemon,
and integrates it in the build and test systems.

The actual functionality of the monitoring daemon is still completely
missing.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

7 years agocfgupgrade: Fix error messages
Bernardo Dal Seno [Wed, 20 Mar 2013 13:24:49 +0000 (14:24 +0100)]
cfgupgrade: Fix error messages

Messages were inconsistent. You could get both an error and a message
saying that everything was fine.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoUpdate "FIXME" string in RAPI
Michele Tartara [Wed, 20 Mar 2013 11:16:47 +0000 (12:16 +0100)]
Update "FIXME" string in RAPI

We are not ready for this change yet. Let's push it to 2.8.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoTypo 'repot' in Server.hs
Helga Velroyen [Wed, 20 Mar 2013 10:42:48 +0000 (11:42 +0100)]
Typo 'repot' in Server.hs

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

7 years agoFix typo 'VolumeN' in bdev.py
Helga Velroyen [Wed, 20 Mar 2013 10:22:57 +0000 (11:22 +0100)]
Fix typo 'VolumeN' in bdev.py

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

7 years agoAdd design document for the "reason trail"
Michele Tartara [Mon, 11 Mar 2013 14:43:57 +0000 (15:43 +0100)]
Add design document for the "reason trail"

This commit adds the design document for introducing "reason trails",
tracing the reason why opcodes are executed, step by step.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoAdd Mond to the list of possible daemons
Michele Tartara [Fri, 8 Mar 2013 17:28:10 +0000 (17:28 +0000)]
Add Mond to the list of possible daemons

Also, add its logfiles and extra log files.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

7 years agoAdd infrastructure for allowing additional logfiles
Michele Tartara [Fri, 8 Mar 2013 14:41:38 +0000 (14:41 +0000)]
Add infrastructure for allowing additional logfiles

Some daemons will need more than the single logfile that is currently
allowed. This patch introduces the infrastructure to allow this.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

7 years agoAdd user and group for the monitoring agent
Michele Tartara [Thu, 21 Feb 2013 16:25:36 +0000 (16:25 +0000)]
Add user and group for the monitoring agent

This commit adds the user and group variables for the monitoring agent in all
the configuration files where they are required.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agorapi client: add target_node to migrate instance
Daniel Krambrock [Tue, 12 Mar 2013 17:17:24 +0000 (18:17 +0100)]
rapi client: add target_node to migrate instance

This allows migrating to any node, as it is already possible for
failover, when instances are externally mirrored.

Signed-off-by: Daniel Krambrock <danielk_lists@z9d.de>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

7 years agoMake diskless instances externally mirrored
Guido Trotter [Tue, 12 Mar 2013 15:07:04 +0000 (16:07 +0100)]
Make diskless instances externally mirrored

This addresses Issue 237.

Mirroring no disk is a no-op. As such we can treat them like mirrored
instances, since the data they need (none) will be present on all nodes.

This is definitely enough to failover or migrate instances with a manual
target node. Further work might be needed for support in hail.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

7 years agoFix migrate/failover -n for ext mirror storage
Guido Trotter [Tue, 12 Mar 2013 15:24:44 +0000 (16:24 +0100)]
Fix migrate/failover -n for ext mirror storage

This fixes issue 396.

- Fix a wrong comment that mentions drbd8 when actually the code acts
  only on externally mirrored instances.
- Fix a wrong assert that requires failover/migrate to acquire the NAL
  on externally mirrored instances: this is the case only when a
  iallocator is used, not if a node is manually specified.

Extra qa should be added, but this can only be done if an externally
mirrored instance or a diskless instance is tested. It is easier to add
it on the master branch, where we have diskless instances support.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

7 years agoAdd a simple tool for checking split-query equivalence
Iustin Pop [Mon, 11 Mar 2013 14:44:17 +0000 (15:44 +0100)]
Add a simple tool for checking split-query equivalence

This is not run automatically (although it could/should), but is very
useful during development.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoEnable split queries for the network
Iustin Pop [Mon, 11 Mar 2013 11:53:36 +0000 (12:53 +0100)]
Enable split queries for the network

Now that all fields are implemented, and (I think) behave equivalent
to the Python implementation, we can switch on the split queries for
networks.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoAdd missing external_reservations query field in confd
Iustin Pop [Mon, 11 Mar 2013 11:52:16 +0000 (12:52 +0100)]
Add missing external_reservations query field in confd

Based on the implemented Ip4Network/Address types, we can now compute
the (external) reservations.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoAdd simple Ip4Address/Ip4Network types
Iustin Pop [Mon, 11 Mar 2013 11:50:38 +0000 (12:50 +0100)]
Add simple Ip4Address/Ip4Network types

This patch adds some very simple IPv4 address/network types, and uses
them in the 'Network' config object.

We need these in order to properly compute the reserved IP addresses,
without depending on an external library (which I haven't found, by
the way). Currently the only operation supported is 'get next IP
address', which is enough for us.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoChange to CamelCase format in Test/Ganeti/Objects.hs
Iustin Pop [Mon, 11 Mar 2013 14:37:15 +0000 (15:37 +0100)]
Change to CamelCase format in Test/Ganeti/Objects.hs

This is in preparation for using OverloadedString extensions in this
file, which conflicts with hlint 1.8.28's handling of annotations.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoExpand TestHelper to allow non-underscore prefixes
Iustin Pop [Mon, 11 Mar 2013 14:34:59 +0000 (15:34 +0100)]
Expand TestHelper to allow non-underscore prefixes

HLint 1.8.28 requires us to always add "ignore CamelCase", which can
be problematic sometimes (e.g. when using OverloadedStrings). Let's
expand TestHelper to also support less-standard 'caseFooBar' names, so
that we can remove the annotations.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoA few style fixes in Ganeti.Network
Iustin Pop [Mon, 11 Mar 2013 11:43:00 +0000 (12:43 +0100)]
A few style fixes in Ganeti.Network

Side-effects of working on some other network-related stuff…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoChange node disk/hv_state query in confd
Iustin Pop [Mon, 11 Mar 2013 10:26:32 +0000 (11:26 +0100)]
Change node disk/hv_state query in confd

Currently, the Python code returns either FS_UNAVAIL (if these
attributes are None) or the proper dicts. As we don't allow editing of
these attributes, in most cases they will therefore be FS_UNAVAIL on
the client.

The Haskell code however returns missingRuntime, which is FS_NODATA,
so a mismatch (that also incurs a RPC call). Fixing it is easy, and
makes the output consistent.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoSort instance list in gnt-group list -opinst_list
Iustin Pop [Mon, 11 Mar 2013 10:20:29 +0000 (11:20 +0100)]
Sort instance list in gnt-group list -opinst_list

The Python code currently sorts this, but the Haskell code not.

This should maybe have a test, but I'm not sure how far we want to
encode such properties in tests… (and the real reason I'm not adding
one is that we don't have a way to generate a random cluster _with_
instances).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoMake gnt-node list -o(p|s)inst_list output stable
Iustin Pop [Mon, 11 Mar 2013 10:17:46 +0000 (11:17 +0100)]
Make gnt-node list -o(p|s)inst_list output stable

Currently, both the Python and Haskell code return the internal
instance list unsorted, which means the output can vary depending on
the phase of the moon (well, the Haskell code actually uses internally
a tree, sorted by the instance name, but it's implementation detail).

By adding an explicit sort step, we guarantee both stability of the
output and consistency between the two query paths.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoFix gnt-backup list -o node via confd
Iustin Pop [Mon, 11 Mar 2013 10:08:56 +0000 (11:08 +0100)]
Fix gnt-backup list -o node via confd

Currently, the 'node' field is declared as a simple config field, so
when only selecting this fields, the runtime gathering is no longer
run and it's presumed that all nodes have a backup. So the output is
not truthful (instead of just listing the nodes with at least one
export, all are listed).

By simply making it a runtime field, filtering works properly and we
get the same output as the masterd code.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoIntroduce better item getter helper in query.py
Iustin Pop [Mon, 11 Mar 2013 09:13:24 +0000 (10:13 +0100)]
Introduce better item getter helper in query.py

And also use it for simple network fields. Currently the Python
library returns jsnull, which is formatted as 'None' by the Python
client, which (IMHO) is not nice, since these fields are strings (some
of them) or "other". An undeclared gateway is easier to diagnose when
properly reported by the server as missing, rather than "existing but
null".

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoFix network query field types/names in the Haskell code
Iustin Pop [Fri, 8 Mar 2013 16:31:44 +0000 (17:31 +0100)]
Fix network query field types/names in the Haskell code

The headers/type/descriptions had some differences from the Python
code, when checked for exact equivalence.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

7 years agoUpdate design doc about free space reporting
Helga Velroyen [Thu, 21 Feb 2013 10:21:46 +0000 (11:21 +0100)]
Update design doc about free space reporting

This patch updates the design doc "Design correct reporting of storage
free space". The modifications were chosen to not conflict with any
future changes of Ganeti regarding storage pools.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>