ganeti-github.git
11 years agoMerge commit 'origin/next' into branch-2.1
Iustin Pop [Thu, 13 Aug 2009 11:15:33 +0000 (13:15 +0200)]
Merge commit 'origin/next' into branch-2.1

* commit 'origin/next': (28 commits)
  Fix a typo in InitCluster
  Ignore results from drained nodes in iallocator
  Ship the ethers hook
  Ethers hook, compatibility with old lockfile
  Remove a few unused imports from noded/masterd
  Move HVM's device_model to a hypervisor parameter
  Implement the KERNEL_PATH parameter for xen-hvm
  Upgrade be/hv params with default values
  Add cluster-init --no-etc-hosts parameter
  objects: add configuration upgrade system
  Update NEWS and version for 2.0.3 release
  example ethers hook: use lockfile-progs
  ethers hook lock: use logger not echo
  ethers hook: reduce the probability of data loss
  devel/upload: revert rsync -p
  export: add meaningful exit code
  Fix detecting of errors in export
  Implement gnt-cluster check-disk-sizes
  rpc: add rpc call for getting disk size
  bdev: Add function for reading actual disk size
  ...

Conflicts:
daemons/ganeti-masterd   - trivial, kept 2.1 version
lib/bootstrap.py         - trivial, kept 2.1 version
lib/cmdlib.py            - integrated the 2.0.3 changes
lib/constants.py         - trivial
lib/hypervisor/hv_xen.py - trivial, kept 2.1 version
lib/objects.py           - trivial, kept 2.1 version
lib/opcodes.py           - integrated the 2.0.3 changes

Signed-off-by: Iustin Pop <iustin@google.com>

11 years agognt-instance replace-disks: Add new --auto option
Michael Hanselmann [Wed, 12 Aug 2009 15:51:24 +0000 (17:51 +0200)]
gnt-instance replace-disks: Add new --auto option

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agocmdlib: Add new automatic disk replacement mode
Michael Hanselmann [Thu, 13 Aug 2009 09:41:04 +0000 (11:41 +0200)]
cmdlib: Add new automatic disk replacement mode

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoTLReplaceDisks: Simplify argument checking logic
Michael Hanselmann [Tue, 11 Aug 2009 16:29:10 +0000 (18:29 +0200)]
TLReplaceDisks: Simplify argument checking logic

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoFix a typo in InitCluster
Guido Trotter [Mon, 8 Jun 2009 13:23:52 +0000 (14:23 +0100)]
Fix a typo in InitCluster

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit 022c3a0b36cb60644b6861ff27ad59202883963c)

11 years agoIgnore results from drained nodes in iallocator
Iustin Pop [Wed, 12 Aug 2009 12:22:41 +0000 (14:22 +0200)]
Ignore results from drained nodes in iallocator

Since drained nodes could be (partially or fully) broken in iallocator,
we ignore results from these nodes when building the cluster map in
preparation for sending it to the script.

This is a cheap change for the stable branch; ideally we should not
query them at all.

The patch also fixes a typo in iallocator.rst.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agoInstanceIpToNodePrimaryIpQuery
Guido Trotter [Tue, 11 Aug 2009 15:30:12 +0000 (16:30 +0100)]
InstanceIpToNodePrimaryIpQuery

This implements CONFD_REQ_NODE_PIP_BY_INSTANCE_IP, and maps instance ips
to primary node ips.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agoquerylib: abstract a couple of common errors
Guido Trotter [Tue, 11 Aug 2009 15:29:10 +0000 (16:29 +0100)]
querylib: abstract a couple of common errors

Rather than composing them every time, we'll keep "ready-made" tuples
for unknown entries and internal errors (which is, yet, unused)

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agoSimpleConfigReader: add IP address lookup helpers
Guido Trotter [Fri, 17 Jul 2009 12:53:37 +0000 (14:53 +0200)]
SimpleConfigReader: add IP address lookup helpers

Add the following functions to SimpleConfigReader:
GetInstanceByIp, nic ip -> instance name
GetNodePrimaryIp, node name -> node primary ip
GetInstancePrimaryNode, instance name -> primary node

For the first one we precalculate the _ip_to_instance, so we don't have
to loop over all instances for each query.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agoAdd NodeRoleQuery
Guido Trotter [Tue, 11 Aug 2009 09:35:50 +0000 (10:35 +0100)]
Add NodeRoleQuery

This implements CONFD_REQ_NODE_ROLE_BYNAME.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agoSimpleConfigReader.GetNodeStatusFlags
Guido Trotter [Tue, 11 Aug 2009 09:34:44 +0000 (10:34 +0100)]
SimpleConfigReader.GetNodeStatusFlags

Adding a way to retrieve the three node status flags, all in one go,
from SimpleConfigReader. Non-existing nodes are handled by returning
"None".

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoAdd a common error code for confd
Guido Trotter [Tue, 11 Aug 2009 09:37:51 +0000 (10:37 +0100)]
Add a common error code for confd

If we standardize common error codes, clients can make use of them to
react, rather than trying to parse error strings.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoAdd a missing node role
Guido Trotter [Tue, 11 Aug 2009 09:33:08 +0000 (10:33 +0100)]
Add a missing node role

Apart from being a master, a candidate, offline or drained, a node can
have one more state: none of those. Adding back the normal, nice,
healthy "just" a node! :)

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoConfdProcessor: move verification to __init__
Guido Trotter [Mon, 10 Aug 2009 16:17:39 +0000 (17:17 +0100)]
ConfdProcessor: move verification to __init__

During a query execution we currently verify whether we have that query
supported in our dispatch table. Since it's an error not to have it
anyway, we move this verification to the class creation, erroring out
early if such a problem exists.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoImplement confd queries
Guido Trotter [Mon, 10 Aug 2009 16:09:20 +0000 (17:09 +0100)]
Implement confd queries

We'll start with a simple ping query, which returns ok if it gets no
query argument, and an error if it gets any. Also non-implemented query
are handled by the base ConfdQuery class, so we don't have to
special-case them in ConfdProcessor anymore.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoAdd constants for a ping confd request
Guido Trotter [Mon, 10 Aug 2009 10:48:32 +0000 (11:48 +0100)]
Add constants for a ping confd request

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoConfdProcessor fix __init__ docstring
Guido Trotter [Mon, 10 Aug 2009 15:41:42 +0000 (16:41 +0100)]
ConfdProcessor fix __init__ docstring

The message is an error deriving from some code moving/refactoring
happening before the initial submission

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoRemove unused imports from confd files
Guido Trotter [Mon, 10 Aug 2009 15:21:39 +0000 (16:21 +0100)]
Remove unused imports from confd files

confd.server and daemons/ganeti-confd import a few modules they don't
actually use. Clean them up.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoDesign-doc: update RedistributeConfig changes
Luca Bigliardi [Wed, 12 Aug 2009 09:03:17 +0000 (10:03 +0100)]
Design-doc: update RedistributeConfig changes

Rewrite the part which discusses the global list of files to be distributed
updating with the description of what is going to be implemented. Just read
below..

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agorapi: Add /2/nodes/[node_name]/storage/modify resource
Michael Hanselmann [Tue, 11 Aug 2009 13:49:44 +0000 (15:49 +0200)]
rapi: Add /2/nodes/[node_name]/storage/modify resource

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agorlib2: Import rapi
Michael Hanselmann [Tue, 11 Aug 2009 13:49:12 +0000 (15:49 +0200)]
rlib2: Import rapi

This fixes ganeti-rapi which was broken in commit 7a95a954.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agorapi: Add /2/nodes/[node_name]/storage resource
Michael Hanselmann [Fri, 7 Aug 2009 13:28:47 +0000 (15:28 +0200)]
rapi: Add /2/nodes/[node_name]/storage resource

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agoShip the ethers hook
Guido Trotter [Mon, 10 Aug 2009 16:40:08 +0000 (17:40 +0100)]
Ship the ethers hook

doc/examples/hooks/ethers has been added without being shipped in the
released tarball. Putting a stop to this.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoConstants: config directories cleanup
Luca Bigliardi [Tue, 11 Aug 2009 09:35:38 +0000 (10:35 +0100)]
Constants: config directories cleanup

Use variables already defined.
Introduce ganeti CONF_DIR.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoSsconf IP lists: change separator
Luca Bigliardi [Tue, 11 Aug 2009 08:25:26 +0000 (09:25 +0100)]
Ssconf IP lists: change separator

Use " " instead of ",".

Signed-off-by: Luca Bigliardi <shammash@google.com>

11 years agoMap Node/IP in ssconf IP lists
Luca Bigliardi [Mon, 10 Aug 2009 18:24:04 +0000 (19:24 +0100)]
Map Node/IP in ssconf IP lists

Map the node hostname with its primary/secondary ganeti IP in ssconf.
In this way a node can discover which are its IPs simply grepping ssconf files,
thus avoiding checks depending on network configuration.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agoEthers hook, compatibility with old lockfile
Luca Bigliardi [Mon, 10 Aug 2009 16:41:14 +0000 (17:41 +0100)]
Ethers hook, compatibility with old lockfile

Remove "-l" option since some ancient systems ship a version of lockfile-progs
not supporting it.

Signed-off-by: Luca Bigliardi <shammash@google.com>

11 years agoRemove a few unused imports from noded/masterd
Guido Trotter [Mon, 10 Aug 2009 15:30:21 +0000 (16:30 +0100)]
Remove a few unused imports from noded/masterd

Signed-off-by: Guido Trotter <ultrotter@google.com>

11 years agoinit script: handle ganeti-confd as well
Guido Trotter [Mon, 3 Aug 2009 17:13:44 +0000 (18:13 +0100)]
init script: handle ganeti-confd as well

Adding ganeti-confd to the example init script. The daemon itself will
only start on master candidates, but the init script can just try to
start it on all nodes.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoInitial confd implementation
Guido Trotter [Wed, 22 Jul 2009 17:09:52 +0000 (18:09 +0100)]
Initial confd implementation

ganeti-confd is a simple asynchronous daemon, which listens on a UDP
port, passes each packet to a processor, and sends back to the client
the result.

It also listens on an inotify socket, in order to reload its
configuration when the ganeti config file changes.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoconfd.server.ConfdProcessor
Guido Trotter [Sat, 8 Aug 2009 13:25:58 +0000 (14:25 +0100)]
confd.server.ConfdProcessor

A ConfdProcessor is the job processor for a confd query.
It's responsible for all the processing, from unpacking it, verifying
it, to calculating the result.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoAdd empty ganeti.confd module
Guido Trotter [Sat, 8 Aug 2009 13:20:28 +0000 (14:20 +0100)]
Add empty ganeti.confd module

It will be used for the confd server code, and for a basic client
library.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoconstants for the confd daemon
Guido Trotter [Sat, 8 Aug 2009 13:18:55 +0000 (14:18 +0100)]
constants for the confd daemon

Adding the various basic daemon constants for confd.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoReply status constants for confd
Guido Trotter [Sat, 8 Aug 2009 13:17:58 +0000 (14:17 +0100)]
Reply status constants for confd

These are valid values for the "status" field in objects.ConfdReply

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoBasic exceptions used by confd
Guido Trotter [Sat, 8 Aug 2009 13:16:40 +0000 (14:16 +0100)]
Basic exceptions used by confd

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoSimpleConfigReader.Reload, avoid double reloads
Guido Trotter [Mon, 3 Aug 2009 17:32:15 +0000 (18:32 +0100)]
SimpleConfigReader.Reload, avoid double reloads

If for some reason we try to call Reload twice but the config file
hasn't changed, we don't want to load it again. Use stat() on the file
to cheaply avoid that situation.

We also return a boolean value, according to whether we have reloaded
the config or not.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoSimpleConfigReader: s/Reload/_Load/
Guido Trotter [Sun, 9 Aug 2009 10:08:40 +0000 (11:08 +0100)]
SimpleConfigReader: s/Reload/_Load/

Since we use it also to load the config file initially we rename the
Reload function to _Load. Then we re-export it as Reload to clients
(since for them it's a reload).

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoRun post cluster initialization LU
Luca Bigliardi [Mon, 10 Aug 2009 08:59:50 +0000 (09:59 +0100)]
Run post cluster initialization LU

Call PostInitCluster from 'gnt-cluster init'.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoPost cluster initialization LU
Luca Bigliardi [Mon, 10 Aug 2009 08:48:48 +0000 (09:48 +0100)]
Post cluster initialization LU

Add an 'empty' logical unit to run hooks after cluster initialization.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoMove HVM's device_model to a hypervisor parameter
Iustin Pop [Thu, 14 May 2009 15:00:32 +0000 (17:00 +0200)]
Move HVM's device_model to a hypervisor parameter

This moves yet another hardcoded value to a hypervisor parameter. I
removed the 64/32 difference as it doesn't seem valid to me - it's more
of a local site config rather than arch config.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
(cherry picked from commit 09ea8710e47288e73746698c50f328e400c056c9)

11 years agoImplement the KERNEL_PATH parameter for xen-hvm
Iustin Pop [Thu, 14 May 2009 12:58:32 +0000 (14:58 +0200)]
Implement the KERNEL_PATH parameter for xen-hvm

For the xen-hvm hypervisor, the KERNEL_PATH parameter is needed but
today is hardcoded to a constants in the xen hypervisor library (argh!).

This patch moves this to a hypervisor constant with the default value
being the current hardcoded path. This will allow cluster/instance
customisation based on the installed xen version.

This should fix Debian bug #528618.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
(cherry picked from commit e2ee1cea7709a2ef82153ec808c4fc3a5bce3ea1)

11 years agoUpgrade be/hv params with default values
Guido Trotter [Thu, 28 May 2009 09:25:36 +0000 (10:25 +0100)]
Upgrade be/hv params with default values

From time to time we're adding new be or hv parameters. With this patch
missing parameters get set to the default value when loading the cluster
object. This patch version also considers the case when hv/be params
don't exist at all, and fixes a broken unit test triggered in that
case.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit c1b42c18b914aa7ea650362ade7489448f71a523)

11 years agoAdd cluster-init --no-etc-hosts parameter
Guido Trotter [Thu, 28 May 2009 09:12:58 +0000 (10:12 +0100)]
Add cluster-init --no-etc-hosts parameter

If --no-etc-hosts is passed in at cluster init time we set a new
parameter in the cluster's object to false, and avoid adding nodes to
the hosts file. The UpgradeConfig function is used to set the value to
True, when upgrading from an old configuration version.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit b86a6bcd476e420269a24a9b3d6289bebba69442)

11 years agoobjects: add configuration upgrade system
Guido Trotter [Thu, 28 May 2009 09:10:11 +0000 (10:10 +0100)]
objects: add configuration upgrade system

Add a very basic configuration update mechanism to objects.
An object can define the UpgradeConfig method, which will be called at
init time, and use it to fill in missing defaults in the configuration.
In the future we may want to make it more complex, for example adding
the config version, but for now a basic solution will do.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
(cherry picked from commit 560428be4f5852813972cd2791f425cf708ca7c6)

11 years agodesign-2.1: Change OS Flavours with OS Variants
Guido Trotter [Fri, 7 Aug 2009 12:42:32 +0000 (13:42 +0100)]
design-2.1: Change OS Flavours with OS Variants

This is the terminology used by libvirt, it's shorter, clearer, and it
makes sense. :)

Also the final sentence about "possible futures" is changed to be a bit
more clear.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoSimpleConfigReader: add serial numbers reading
Guido Trotter [Thu, 6 Aug 2009 17:38:08 +0000 (18:38 +0100)]
SimpleConfigReader: add serial numbers reading

GetConfigSerialNo and GetClusterSerialNo return respectively the serial
number for the config and for the cluster object.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agoserializer.DumpSignedJson
Guido Trotter [Fri, 7 Aug 2009 10:07:37 +0000 (11:07 +0100)]
serializer.DumpSignedJson

Don't indent the final message.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agoconstants: confd node roles
Guido Trotter [Thu, 6 Aug 2009 17:15:15 +0000 (18:15 +0100)]
constants: confd node roles

confd will return the node role as an integer, which represents one of
the mutually exclusive roles a node can be in.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoMerge branch 'master' into next
Guido Trotter [Fri, 7 Aug 2009 11:22:16 +0000 (12:22 +0100)]
Merge branch 'master' into next

* master:
  Update NEWS and version for 2.0.3 release
  devel/upload: revert rsync -p
  export: add meaningful exit code
  Fix detecting of errors in export
  Implement gnt-cluster check-disk-sizes
  rpc: add rpc call for getting disk size
  bdev: Add function for reading actual disk size
  Implement --ignore-size in activate-disks
  Add ignore size support in _AssembleInstanceDisks
  Add a objects.Disk.UnsetSize() method
  bdev: allow ignoring of size in Assemble()
  Fix instance import net option
  Simplify the devel/upload script
  Add a Copy method to object.ConfigObject
  Extend call_node_start_master rpc with no_voting

Conflicts:

  daemons/ganeti-masterd
    s/SimpleConfigReader/SimpleStore/ VS start-master no-voting
    (kept both)

Signed-off-by: Guido Trotter <ultrotter@google.com>

11 years agoconstants: confd query types
Guido Trotter [Thu, 6 Aug 2009 17:08:34 +0000 (18:08 +0100)]
constants: confd query types

Initially confd will support only two queries:

CONFD_REQ_NODE_ROLE_BYNAME
Given a node name, return its role.

CONFD_REQ_NODE_PIP_BY_INSTANCE_IP
Given an instance ip, return its node primary ip.
This rather weird query is the basis for ganeti nbma lookup.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agodesign-2.1: detail confd wire protocol
Guido Trotter [Fri, 7 Aug 2009 10:05:04 +0000 (11:05 +0100)]
design-2.1: detail confd wire protocol

Until now it was being kept too vague, so here we give some real
examples of how things are going to be.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agoUpdate NEWS and version for 2.0.3 release v2.0.3
Iustin Pop [Fri, 7 Aug 2009 09:23:16 +0000 (11:23 +0200)]
Update NEWS and version for 2.0.3 release

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agoSimpleConfigReader.Reload()
Guido Trotter [Mon, 3 Aug 2009 15:09:28 +0000 (16:09 +0100)]
SimpleConfigReader.Reload()

Rather than initializing the config statically at class creation time,
we load it every time Reload() is called.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoConfd{Request,Reply} objects
Guido Trotter [Wed, 5 Aug 2009 12:30:51 +0000 (13:30 +0100)]
Confd{Request,Reply} objects

These objects are used to store confd queries and replies.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoSerializer, remove salt_verifier functionality
Guido Trotter [Thu, 6 Aug 2009 13:44:58 +0000 (14:44 +0100)]
Serializer, remove salt_verifier functionality

The salt needs to be returned anyway, so we don't have to add another
key for the sender to recognize which request an answer is answering, so
all that infrastructure is useless. :(

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agopyinotify: configure checks and documentation
Guido Trotter [Thu, 6 Aug 2009 09:45:01 +0000 (10:45 +0100)]
pyinotify: configure checks and documentation

After 74d519e3b91845a17ae095eb7d58dd9e3d1303e8 Ganeti depends on
pyinotify. Updating the documentation accordingly and checking for its
presence at configure time.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoasycnotifier.AsyncNotifier
Guido Trotter [Thu, 6 Aug 2009 09:18:41 +0000 (10:18 +0100)]
asycnotifier.AsyncNotifier

AsyncNotifier is a special asyncore class that delivers inotify events
asynchronously.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoSimpleConfigReader: Handle errors when loading
Guido Trotter [Fri, 17 Jul 2009 12:52:31 +0000 (14:52 +0200)]
SimpleConfigReader: Handle errors when loading

Handling both IOErrors and ValueErrors (thrown by the simplejson loader)

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agossconf.CheckMasterCandidate
Guido Trotter [Tue, 14 Jul 2009 10:35:04 +0000 (12:35 +0200)]
ssconf.CheckMasterCandidate

This function checks that the current node is a master candidate, and
terminates otherwise. It will be used upon ganeti-confd startup.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoConvert ldisk_degraded to tri-state value
Michael Hanselmann [Thu, 6 Aug 2009 11:26:14 +0000 (13:26 +0200)]
Convert ldisk_degraded to tri-state value

This allows us to report “uncertain” states (LDS_UNKNOWN) for cases
where the code can't easily detect or report what's wrong with a
block device.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoAdd constants for local disk status
Michael Hanselmann [Thu, 6 Aug 2009 11:28:16 +0000 (13:28 +0200)]
Add constants for local disk status

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoHandle None result from BlockdevFind
Michael Hanselmann [Thu, 6 Aug 2009 11:27:27 +0000 (13:27 +0200)]
Handle None result from BlockdevFind

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoobjects.BlockDevStatus: Remove ToLegacyStatus
Michael Hanselmann [Wed, 5 Aug 2009 11:21:27 +0000 (13:21 +0200)]
objects.BlockDevStatus: Remove ToLegacyStatus

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoexample ethers hook: use lockfile-progs
Guido Trotter [Wed, 5 Aug 2009 23:02:17 +0000 (00:02 +0100)]
example ethers hook: use lockfile-progs

Rather than writing our own locking routing, use the one implemented by
the lockfile-create program.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoethers hook lock: use logger not echo
Luca Bigliardi [Wed, 5 Aug 2009 17:58:42 +0000 (18:58 +0100)]
ethers hook lock: use logger not echo

Overwrite debugging 'echo's

Signed-off-by: Luca Bigliardi <shammash@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agoethers hook: reduce the probability of data loss
Luca Bigliardi [Wed, 5 Aug 2009 17:48:20 +0000 (18:48 +0100)]
ethers hook: reduce the probability of data loss

The hook was exiting immediately if lock was not acquired, entering a timed
loop to have more chances when acquiring the lock.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agoAdd master candidates IPs informations to ssconf
Luca Bigliardi [Wed, 5 Aug 2009 16:31:38 +0000 (17:31 +0100)]
Add master candidates IPs informations to ssconf

This will be used when querying confd, in order not to rely on DNS being
available.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agoTestParameterNames: also check nic parameters
Guido Trotter [Wed, 5 Aug 2009 16:39:41 +0000 (17:39 +0100)]
TestParameterNames: also check nic parameters

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agoConfigObject.ToDict() only export non-None values
Guido Trotter [Wed, 5 Aug 2009 12:46:43 +0000 (13:46 +0100)]
ConfigObject.ToDict() only export non-None values

The method is changed to a normal loop, to avoid calling getattr()
twice. Also __getstate__ is changed to just use ToDict() by default.

This should also make __getstate__ work for objects which have to
override the ToDict function because they contain other objects.

__setstate__ is probably still broken in this case, but so it was
before, and it's not used inside our code, so I'll pretend not to have
noticed, as there is no "nice" way to fix it, without overriding it all
over the place :(

Some unittests are added as a bonus, to make sure we behave well.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoAdd nodes IPs informations to ssconf
Luca Bigliardi [Wed, 5 Aug 2009 13:44:49 +0000 (14:44 +0100)]
Add nodes IPs informations to ssconf

Having a list of primary/secondary IPs of all the nodes in ssconf can be useful
for scripts/hooks which need to automatically configure network properties for
the whole cluster (e.g.: ipsec/netfilter rules) without relying on a
working DNS.

Signed-off-by: Luca Bigliardi <shammash@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agodevel/upload: revert rsync -p
Iustin Pop [Wed, 5 Aug 2009 13:29:16 +0000 (15:29 +0200)]
devel/upload: revert rsync -p

The permissions replications also will change the permissions on the /
and /usr directories, which is bad. This reverts it to the original
behaviour.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agoserializer: fix a few docstrings
Guido Trotter [Wed, 5 Aug 2009 10:17:17 +0000 (11:17 +0100)]
serializer: fix a few docstrings

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agoexport: add meaningful exit code
Iustin Pop [Wed, 5 Aug 2009 11:02:48 +0000 (13:02 +0200)]
export: add meaningful exit code

Currently ‘gnt-backup export’ always returns exit code zero, even in the
face of complete failure during backup (only failure to stop/start the
instance will cause job failure and thus non-zero exit code). This is
bad, since one cannot script the backup.

This patch adds some simple results from the LU so that the command line
script can return good exit code. It will:
  - return zero for full success (snapshot removal errors are ignored
    though)
  - return one for full failure (finalize export failure or all disks
    failure)
  - return two for partial failure (some disks backed up, some not)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agoFix detecting of errors in export
Iustin Pop [Wed, 5 Aug 2009 09:49:56 +0000 (11:49 +0200)]
Fix detecting of errors in export

This should fix issue 61, by explicitely calling bash (which is is now a
non-explicit dependency) and setting the pipefail command.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agoUse objects for blockdev_getmirrorstatus RPC call result
Michael Hanselmann [Tue, 4 Aug 2009 16:22:27 +0000 (18:22 +0200)]
Use objects for blockdev_getmirrorstatus RPC call result

This patch changes the return type for backend.BlockdevGetmirrorstatus from
a list of tuples to a list of objects.BlockDevStatus instances.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoUse object for blockdev_find RPC call result
Michael Hanselmann [Tue, 4 Aug 2009 16:21:13 +0000 (18:21 +0200)]
Use object for blockdev_find RPC call result

This patch changes the return type for backend.BlockdevFind to an object
(objects.BlockDevStatus). Before a tuple was used. Adding more values to
this tuple causes a lot of work.  Converting the result to an object with
properties will make this a bit simpler.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agoImplement gnt-cluster check-disk-sizes
Iustin Pop [Tue, 4 Aug 2009 11:14:13 +0000 (13:14 +0200)]
Implement gnt-cluster check-disk-sizes

This patch adds a new opcode and lu for checking disk sizes. Currently
it does only top-level disk verification, and also doesn't check
primary/secondary node size mismatches (these two are added as TODOs in
the Exec() function of the LU).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agorpc: add rpc call for getting disk size
Iustin Pop [Tue, 4 Aug 2009 10:43:22 +0000 (12:43 +0200)]
rpc: add rpc call for getting disk size

Note that this exports the disk size as bdev returns it, in bytes. The
value will be converted to MiB in cmdlib.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agobdev: Add function for reading actual disk size
Iustin Pop [Sun, 26 Jul 2009 15:09:26 +0000 (17:09 +0200)]
bdev: Add function for reading actual disk size

This patch adds a GetActualSize for block devices that returns the
actual disk size. It is done using blockdev (and stat for file storage).

While this could be done via reading /sys/block/N/size, that is not as
simple as running blockdev, as the correspondence between an LV and its
sys entry is not straightforward.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agoImplement --ignore-size in activate-disks
Iustin Pop [Sun, 26 Jul 2009 19:35:10 +0000 (21:35 +0200)]
Implement --ignore-size in activate-disks

This patch modified OpActivateDisks, LUActivateDisks and gnt-instance
activate-disks to support and pass this option to
_AssembleInstanceDisks.

The patch is quite trivial I think; there should be no issues from it
except if used when not needed.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agoAdd ignore size support in _AssembleInstanceDisks
Iustin Pop [Sun, 26 Jul 2009 19:19:15 +0000 (21:19 +0200)]
Add ignore size support in _AssembleInstanceDisks

This patch adds an optional parameter to _AssembleInstanceDisks that
allows ignoring of size information by making a copy of the disk
structure and setting the size to zero.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agoAdd a objects.Disk.UnsetSize() method
Iustin Pop [Sun, 26 Jul 2009 19:26:57 +0000 (21:26 +0200)]
Add a objects.Disk.UnsetSize() method

This method recursively resets the size of the disk and its children to
zero.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agobdev: allow ignoring of size in Assemble()
Iustin Pop [Sun, 26 Jul 2009 15:27:49 +0000 (17:27 +0200)]
bdev: allow ignoring of size in Assemble()

This patch changes the DRBD8 class (the only one to use the size in
Assemble) to ignore the size in Assemble when a zero size is passed.
This will allow activation of disks even when the size recorded in the
configuration is wrong.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agoFix instance import net option
Iustin Pop [Tue, 4 Aug 2009 12:33:37 +0000 (14:33 +0200)]
Fix instance import net option

This is identical to dc30b0e4 but applied to gnt-backup. Thanks to user
ocaner for catching it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agoSimplify the devel/upload script
Iustin Pop [Tue, 4 Aug 2009 11:38:42 +0000 (13:38 +0200)]
Simplify the devel/upload script

Instead of multiple uploads to each node, this script copies everything
as needed to the temporary directory, exactly as to be installed in the
destination machine, then runs only one rsync per host.

This is more dangerous (we can break /etc now), but for development
machines is fine.

The patch then also uploads the bash completions and the current name
for the cron job (I think that ganeti-master-cron is a deprecated name,
not that someone actually intends to upload a file named like that). A
flag --no-cron is added to skip uploading the cron file if desired.

The patch also changes rsync to propagate the file permissions.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agognt-node physical-volumes: Add storage type parameter
Michael Hanselmann [Tue, 4 Aug 2009 11:31:45 +0000 (13:31 +0200)]
gnt-node physical-volumes: Add storage type parameter

This way the user can also show storage types other than lvm-pv.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agocmdlib: Fix parameters for storage.FileStorage
Michael Hanselmann [Tue, 4 Aug 2009 11:30:32 +0000 (13:30 +0200)]
cmdlib: Fix parameters for storage.FileStorage

It wants a list of directories, not a string.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoAdd “gnt-node modify-volume” command
Michael Hanselmann [Thu, 30 Jul 2009 14:44:21 +0000 (16:44 +0200)]
Add “gnt-node modify-volume” command

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agocmdlib: Add opcode to modify storage unit fields
Michael Hanselmann [Thu, 30 Jul 2009 14:38:14 +0000 (16:38 +0200)]
cmdlib: Add opcode to modify storage unit fields

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agoAdd RPC calls to modify storage fields
Michael Hanselmann [Thu, 30 Jul 2009 14:37:29 +0000 (16:37 +0200)]
Add RPC calls to modify storage fields

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agostorage: Add function to modify fields
Michael Hanselmann [Tue, 4 Aug 2009 09:43:03 +0000 (11:43 +0200)]
storage: Add function to modify fields

This allows the “allocatable” flag on LVM PVs to be changed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

11 years agoMerge commit 'origin/branch-2.1' into feature/containers
Iustin Pop [Tue, 4 Aug 2009 08:52:10 +0000 (10:52 +0200)]
Merge commit 'origin/branch-2.1' into feature/containers

* commit 'origin/branch-2.1': (66 commits)
  Add automated disk repair changes to design doc
  Add review script
  Implement “gnt-node physical-volumes” command
  Add new opcode to list physical volumes
  storage: Use constants.py instead of local constants
  storage: Fix semantics for directory size
  Add “gnt-job watch” command
  jqueue: Fix error when WaitForJobChange gets invalid ID
  jqueue: Update message for cancelling running job
  cmdlib: Change tasklet logging to debug level
  rapi: Add /2/nodes/[node_name]/migrate resource
  gnt-node: Use new opcode to migrate node
  cmdlib: Add new opcode to migrate node
  rapi: Add default parameter to _checkIntVariable
  cmdlib: Add logging for tasklets
  cmdlib: Fix tasklets handling if no tasklets are added
  rapi: Add /2/[node_name]/evacuate resource
  Add information about storage units framework
  Add RPC calls for storage unit list
  Add first implementation of generic storage unit framework
  ...

11 years agoAdd automated disk repair changes to design doc
Michael Hanselmann [Mon, 3 Aug 2009 13:22:05 +0000 (15:22 +0200)]
Add automated disk repair changes to design doc

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoAdd review script
Michael Hanselmann [Mon, 3 Aug 2009 13:24:31 +0000 (15:24 +0200)]
Add review script

I've been using this script for a while to update commits before
pushing them to the main repository. It copies all commits in a
range to another branch using git cherry-pick and starts an editor
to modify the Reviewed-by: line(s) for each commit. The script is
certainly not perfect, but it does the job.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoImplement “gnt-node physical-volumes” command
Michael Hanselmann [Wed, 29 Jul 2009 16:01:44 +0000 (18:01 +0200)]
Implement “gnt-node physical-volumes” command

This command can be used to list all physical volumes on nodes.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoAdd new opcode to list physical volumes
Michael Hanselmann [Wed, 29 Jul 2009 16:01:06 +0000 (18:01 +0200)]
Add new opcode to list physical volumes

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agostorage: Use constants.py instead of local constants
Michael Hanselmann [Wed, 29 Jul 2009 13:57:20 +0000 (15:57 +0200)]
storage: Use constants.py instead of local constants

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agostorage: Fix semantics for directory size
Michael Hanselmann [Wed, 29 Jul 2009 13:52:00 +0000 (15:52 +0200)]
storage: Fix semantics for directory size

The actual directory size is "used" space, not the total space on
the filesystem.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

11 years agoAdd a Copy method to object.ConfigObject
Iustin Pop [Sun, 26 Jul 2009 19:46:07 +0000 (21:46 +0200)]
Add a Copy method to object.ConfigObject

This small patch adds a simple Copy method that is can be used for
'throw-away' copies of objects.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

11 years agoMerge branch 'next' into branch-2.1
Michael Hanselmann [Mon, 3 Aug 2009 09:25:06 +0000 (11:25 +0200)]
Merge branch 'next' into branch-2.1

* next:
  Add “gnt-job watch” command
  jqueue: Fix error when WaitForJobChange gets invalid ID
  jqueue: Update message for cancelling running job