ganeti-github.git
4 years agoDocument the increased timeout as an incompatible change
Klaus Aehlig [Wed, 20 Jan 2016 11:13:33 +0000 (12:13 +0100)]
Document the increased timeout as an incompatible change

While the timeout for communication with luxid is mainly
an internal parameter, it also changes which response time
for Ganeti tools is still to be considered normal. Hence
warn users that might have higher level tools interacting
with Ganeti.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoIncrease timeouts for luxi by a factor of 3
Klaus Aehlig [Wed, 20 Jan 2016 11:07:03 +0000 (12:07 +0100)]
Increase timeouts for luxi by a factor of 3

While sending answers lazily as Strings has reduced memory footprint
by over an order of magnitude, it seems that answer times have gotten
slower. Accept this trade off treating time for space and increase all
timeouts accordingly.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoDo not repeat constants in comments
Klaus Aehlig [Wed, 20 Jan 2016 11:02:49 +0000 (12:02 +0100)]
Do not repeat constants in comments

...as this works against the idea of having all constants in one
central place so that they can be changed in a simple way.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoMerge branch 'stable-2.15' into stable-2.16
Klaus Aehlig [Fri, 15 Jan 2016 13:59:47 +0000 (14:59 +0100)]
Merge branch 'stable-2.15' into stable-2.16

* stable-2.15
  Catch IOError of SSH files when removing node
  Fix renew-crypto on one-node-cluster
  ssh_update: log data that is received
  Increase timeout of RPC adding/removing keys
  After TestNodeModify, fix the pool of master candidates

* stable-2.14
  Test disk attachment with different primary nodes
  Check for same primary node before disk attachment
  Add detach/attach sequence test
  Allow disk attachment with external storage

* stable-2.13
  Run ssh-key renewal in debug mode during upgrade

* stable-2.12
  Increase minimal sizes of test online nodes
  Also log the high-level upgrade steps
  Add function to provide logged user feedback
  Run renew-crypto in upgrades in debug mode
  Unconditionally log upgrades at debug level
  Document healthy-majority restriction on master-failover
  Check for healthy majority on master failover with voting
  Add a predicate testing that a majority of nodes is healthy
  Fix outdated comment
  Pass arguments to correct daemons during master-failover
  Fix documentation for master-failover

* stable-2.11
  (no changes)

* stable-2.10
  KVM: explicitly configure routed NICs late

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.14' into stable-2.15
Klaus Aehlig [Fri, 15 Jan 2016 10:17:06 +0000 (11:17 +0100)]
Merge branch 'stable-2.14' into stable-2.15

* stable-2.14
  Test disk attachment with different primary nodes
  Check for same primary node before disk attachment
  Add detach/attach sequence test
  Allow disk attachment with external storage

* stable-2.13
  Run ssh-key renewal in debug mode during upgrade

* stable-2.12
  Increase minimal sizes of test online nodes
  Also log the high-level upgrade steps
  Add function to provide logged user feedback
  Run renew-crypto in upgrades in debug mode
  Unconditionally log upgrades at debug level
  Document healthy-majority restriction on master-failover
  Check for healthy majority on master failover with voting
  Add a predicate testing that a majority of nodes is healthy
  Fix outdated comment
  Pass arguments to correct daemons during master-failover
  Fix documentation for master-failover

* stable-2.11
  (no changes)

* stable-2.10
  KVM: explicitly configure routed NICs late

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.13' into stable-2.14
Klaus Aehlig [Thu, 14 Jan 2016 17:01:17 +0000 (18:01 +0100)]
Merge branch 'stable-2.13' into stable-2.14

* stable-2.13
  Run ssh-key renewal in debug mode during upgrade

* stable-2.12
  Increase minimal sizes of test online nodes
  Also log the high-level upgrade steps
  Add function to provide logged user feedback
  Run renew-crypto in upgrades in debug mode
  Unconditionally log upgrades at debug level
  Document healthy-majority restriction on master-failover
  Check for healthy majority on master failover with voting
  Add a predicate testing that a majority of nodes is healthy
  Fix outdated comment
  Pass arguments to correct daemons during master-failover
  Fix documentation for master-failover

* stable-2.11
  (no changes)

* stable-2.10
  KVM: explicitly configure routed NICs late

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoRun ssh-key renewal in debug mode during upgrade
Klaus Aehlig [Thu, 14 Jan 2016 14:10:01 +0000 (15:10 +0100)]
Run ssh-key renewal in debug mode during upgrade

As errors during an upgrade of Ganeti are harder to
understand, as two versions of Ganeti are involved,
provide more debug information for everything that happens
during that process. Note that upgrades are a rare event,
so we do not have to worry about the size of log files
too much.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoMerge branch 'stable-2.12' into stable-2.13
Klaus Aehlig [Thu, 14 Jan 2016 13:07:02 +0000 (14:07 +0100)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Increase minimal sizes of test online nodes
  Also log the high-level upgrade steps
  Add function to provide logged user feedback
  Run renew-crypto in upgrades in debug mode
  Unconditionally log upgrades at debug level
  Document healthy-majority restriction on master-failover
  Check for healthy majority on master failover with voting
  Add a predicate testing that a majority of nodes is healthy
  Fix outdated comment
  Pass arguments to correct daemons during master-failover
  Fix documentation for master-failover

* stable-2.11
  (no changes)

* stable-2.10
  KVM: explicitly configure routed NICs late

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoIncrease minimal sizes of test online nodes stable-2.12
Klaus Aehlig [Tue, 8 Dec 2015 16:05:10 +0000 (17:05 +0100)]
Increase minimal sizes of test online nodes

A lot of our tests work by generating a node and a
strictly smaller instance and then continue under
the assumption that the instance will fit on the node.
To obtain a strictly smaller instance, we take an instance
of size at most half the free resources of the node. The
problem with this approach is that we also require minimal
resources of an instance (for examples to be realistic); now,
this can lead to an upper bound lower than the lower bound
and, by the way QuickCheck's `choose` works, still a value
between these bounds is chosen, violating the assumptions
about node and instance sizes.

To avoid those problems, set the minimal resources of an
allocatable node so that half of them is still bigger than
the minimal resources of an instance.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

Cherry-picked-from: 6ccf05c1507c58e
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoMerge branch 'stable-2.11' into stable-2.12
Klaus Aehlig [Tue, 12 Jan 2016 15:46:43 +0000 (16:46 +0100)]
Merge branch 'stable-2.11' into stable-2.12

* stable-2.11
  (no changes)

* stable-2.10
  KVM: explicitly configure routed NICs late

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoAlso log the high-level upgrade steps
Klaus Aehlig [Tue, 12 Jan 2016 10:37:13 +0000 (11:37 +0100)]
Also log the high-level upgrade steps

The upgrade of a Ganeti cluster is done in several
high-level steps ("Draining queue", "Pausing the watcher",
"Stopping daemons", ...). Log those headings as well in
order to simplify reading the log file; with these headings,
it is more easy to understand which goal is aimed for with
all the micro-step RunCmd log entries.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoAdd function to provide logged user feedback
Klaus Aehlig [Tue, 12 Jan 2016 14:54:33 +0000 (15:54 +0100)]
Add function to provide logged user feedback

Add a utility function that provides feedback to the
user on stdout that is additionally logged (at INFO level)
in the log file.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoRun renew-crypto in upgrades in debug mode
Klaus Aehlig [Mon, 11 Jan 2016 17:11:43 +0000 (18:11 +0100)]
Run renew-crypto in upgrades in debug mode

As errors during an upgrade of Ganeti are harder to
understand, as two versions of Ganeti are involved,
provide more debug information for everything that happens
during that process. Note that upgrades are a rare event,
so we do not have to worry about the size of log files
too much.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoUnconditionally log upgrades at debug level
Klaus Aehlig [Tue, 12 Jan 2016 09:38:50 +0000 (10:38 +0100)]
Unconditionally log upgrades at debug level

Cluster upgrades to a new minor version of Ganeti are a rare
operation (in fact, new minor versions are released only every
3 months). Therefore, we do not have to worry about increased
size of log files. However, upgrades of Ganeti are complicated
in the sense that, should something break during the upgrade, it
is not immediately obvious, in which Ganeti state is left in. Therefore,
always provide full log information on upgrades. Fixes issue 1137.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoSend messages as Strings
Klaus Aehlig [Mon, 30 Nov 2015 14:19:38 +0000 (15:19 +0100)]
Send messages as Strings

ByteStrings are a more compact representation of a sequence of octets
than are Strings. However, converting a String into a ByteString, even
a lazy one, looks at a huge number of characters before the first goes
out of scope; thus the String gets enforced effectively. As Strings,
as a list of unicode characters, have a quite memory-intense representation,
this loss of lazyness results in a memory spike that is quite significant,
at least for restricted environments like a Xen dom0, when sending the
whole Ganeti configuration.

Therefore, send messages as String over the wire to preserve lazyness.
This is sound, as our JSON representation is 7-bit clean, and hence
every character coincides with its UTF8 encoding. On a larger cluster,
this saved an order of magnitude in peak memory usage.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.10' into stable-2.11 stable-2.11
Klaus Aehlig [Mon, 11 Jan 2016 11:30:30 +0000 (12:30 +0100)]
Merge branch 'stable-2.10' into stable-2.11

* stable-2.10
  KVM: explicitly configure routed NICs late

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoTest disk attachment with different primary nodes
Lisa Velden [Mon, 11 Jan 2016 11:12:49 +0000 (12:12 +0100)]
Test disk attachment with different primary nodes

Test a detach/attach sequence with a DRBD disk and a different primary
node for the disk and the instance. This should raise an exception.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoCheck for same primary node before disk attachment
Lisa Velden [Mon, 11 Jan 2016 11:09:47 +0000 (12:09 +0100)]
Check for same primary node before disk attachment

Make sure a DRBD disk has the same primary node as the instance where it
will be attached to.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoDocument healthy-majority restriction on master-failover
Klaus Aehlig [Fri, 8 Jan 2016 13:51:20 +0000 (14:51 +0100)]
Document healthy-majority restriction on master-failover

The previous patch introduced a behavioral change for master-failover:
it is rejected unless a majority of nodes is healthy or the --no-voting
option is given. (While we in general do not change behavior on a stable
branch, rejecting an operation that can be retried with different command-line
options is better than breaking the cluster completely.) Document this.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoCheck for healthy majority on master failover with voting
Klaus Aehlig [Fri, 8 Jan 2016 10:37:17 +0000 (11:37 +0100)]
Check for healthy majority on master failover with voting

The normal procedure for a master failover is that, after telling
each node the new master, the daemons on the new master node are
started the standard way, i.e., with voting. This, however, requires
that a majority of nodes is still healthy; otherwise, the failover
will result in the daemons not starting and thus a broken cluster.
Therefore, reject master failovers with voting, unless we can verify
that a majority of nodes is still responding.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoAdd a predicate testing that a majority of nodes is healthy
Klaus Aehlig [Fri, 8 Jan 2016 11:26:57 +0000 (12:26 +0100)]
Add a predicate testing that a majority of nodes is healthy

For standard master failover (with voting), it is necessary
that the majority of nodes is still reachable and can answer
questions about which node is master. Add a predicate verifying
that this is still true.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoFix outdated comment
Klaus Aehlig [Fri, 8 Jan 2016 10:54:47 +0000 (11:54 +0100)]
Fix outdated comment

Commit 5e641d0a introduced also counting the vote of
the node itself. Adapt the parameter description
accordingly.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoCatch IOError of SSH files when removing node
Helga Velroyen [Thu, 17 Dec 2015 09:03:17 +0000 (10:03 +0100)]
Catch IOError of SSH files when removing node

This patch catches an IOError when a node is removed
from a cluster and the SSH files of the node are messed
up. Previously, this caused the removal to fail, which
is not exactly what you want when removing a messed
up node.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Cherry-picked-from: a856040abc755b4
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoFix renew-crypto on one-node-cluster
Helga Velroyen [Wed, 16 Dec 2015 10:03:23 +0000 (11:03 +0100)]
Fix renew-crypto on one-node-cluster

There was a bug which made 'gnt-cluster renew-crypto'
crash if it is a one-node cluster. This patch fixes
it by checking if there are any non-master nodes
to update at all.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Cherry-picked-from: 88ac338d88465cc0b
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agossh_update: log data that is received
Helga Velroyen [Tue, 15 Dec 2015 14:03:53 +0000 (15:03 +0100)]
ssh_update: log data that is received

Debugging ssh_update can be annoying, because the data
used as input is not dumped anywhere. This patch logs
makes sure it gets logged (at DEBUG level) when
ssh_update receives the data.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Cherry-picked-from: 5c370ec180
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoKVM: explicitly configure routed NICs late stable-2.10
Apollon Oikonomopoulos [Wed, 2 Dec 2015 12:35:42 +0000 (14:35 +0200)]
KVM: explicitly configure routed NICs late

Commit cc8a8ed7 outlined the reasons for configuring bridged NICs early
during live migration and routed NICs after migration has been finished.
Back then these were the only types of NICs available, however with the
introduction of OVS support this has changed.

Since OVS bridges are essentially bridges, the considerations outlined
in cc8a8ed7 still apply: in particular, we do not want to lose the
gratuitous ARP sent out by the KVM NICs, so we have to configure
the OVS interfaces early in the migration process as well.

Rather than explicitly configure bridged and OVS interfaces early, we
prefer to explicitly configure routed interfaces late, since this leads
to more compact code.

Signed-off-by: Apollon Oikonomopoulos <apoikos@gmail.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoIncrease timeout of RPC adding/removing keys
Helga Velroyen [Thu, 7 Jan 2016 13:27:29 +0000 (14:27 +0100)]
Increase timeout of RPC adding/removing keys

This patch increases the timeout for the RPC calls that
add and remove SSH keys to the cluster. This is necessary,
because in big clusters the distribution/removal of a
key takes too long as Ganeti has to contact every node in
the cluster.

This patch increases the timeout from URGENT to FAST
(the next higher option).

The alternatives to this include splitting up the
RPC call to several calls, which will add addiional
overall runtime and RPC overhead as well as security
implications. Since the higher timeout was tested
in a big cluster, we go with this for now.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoAfter TestNodeModify, fix the pool of master candidates
Klaus Aehlig [Tue, 22 Dec 2015 11:35:40 +0000 (12:35 +0100)]
After TestNodeModify, fix the pool of master candidates

The test TestNodeModify temporarily modifies the cluster parameter
candidate-pool-size, which controls the minimal desirable number of
master candidates. Depending on the size of the test cluster, this
temporary modification can be a decrease (for clusters with up to 10
nodes) or an increase (for clusters with 12 or more nodes). Ganeti's
behavior upon change of the candidate pool size is to promote nodes to
master candidates upon increase, but do nothing upon decrease. This is
a safe behavior, as too many master candidates is not a problem; the
chance of data loss is even smaller. However, it means that the test
has a size effect of, for large test cluster, increasing the actual
number of nodes that are master candidates. While not a problem for
correctness, this side effect does affect our performance tests (which
usually are run after the functional tests) as more master candidates
means more nodes to replicate information to.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoPass arguments to correct daemons during master-failover
Hrvoje Ribicic [Thu, 17 Dec 2015 00:18:50 +0000 (00:18 +0000)]
Pass arguments to correct daemons during master-failover

A master-failover can be executed with the --no-voting flag, making
Ganeti start daemons despite a lack of votes. This is necessary to
fail over a cluster reduced to two nodes. The feature has not
been working since 2.12 daemon refactoring, as the daemon parameters
were passed through environmental variables that were not updated.

This commit passes the parameters correctly, and fixes issue 1159.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoFix typo 'option' instead of 'options'
Helga Velroyen [Tue, 5 Jan 2016 09:49:13 +0000 (10:49 +0100)]
Fix typo 'option' instead of 'options'

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoMerge branch 'stable-2.15' into stable-2.16
Helga Velroyen [Mon, 4 Jan 2016 16:07:50 +0000 (17:07 +0100)]
Merge branch 'stable-2.15' into stable-2.16

* stable-2.15
  Add more documentation to testutils_ssh.py
  renew-crypto: use bulk-removal of SSH keys
  Use bulk-removal of SSH keys for single keys
  Bulk-removing SSH keys of diverse set of nodes
  Bulk-removal of SSH keys of normal nodes
  Bulk-remove SSH keys of potential master candidates
  Bulk-removal of SSH keys
  testutils: add keys to own 'authorized_keys' file
  Make mock SSH file manager deal with lists
  Don't deepcopy the config if the old value is not needed
  Revision bump for 2.15.2
  Update NEWS file for 2.15.2
  Compute lock allocation strictly

* stable-2.14
  Revision bump for 2.14.2
  Update NEWS file for 2.14.2
  Fix lines with more than 80 characters
  Add more detach/attach sequence tests
  Allow disk attachment to diskless instances
  Improve tests for attaching disks

* stable-2.13
  Revision bump for 2.13.3
  Update NEWS file for 2.13.3

* stable-2.12
  Bump revision number for 2.12.6
  Update NEWS file for 2.12.6
  Restrict showing of DRBD secret using types
  Calculate correct affected nodes set in InstanceChangeGroup

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  NEWS
  configure.ac

Resolutions:
  NEWS: merge contents in right order
  configure.ac: keep version number of 2.16

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoFix documentation for master-failover
Hrvoje Ribicic [Mon, 4 Jan 2016 13:16:45 +0000 (14:16 +0100)]
Fix documentation for master-failover

The gnt-cluster manual still specified that arguments should be passed
to the master daemon - one which no longer exists. This patch specifies
the two new daemons to which arguments should be passed instead.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoAdd detach/attach sequence test
Lisa Velden [Wed, 16 Dec 2015 15:27:44 +0000 (16:27 +0100)]
Add detach/attach sequence test

Add a test for external storage.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoAllow disk attachment with external storage
Lisa Velden [Wed, 16 Dec 2015 13:57:43 +0000 (14:57 +0100)]
Allow disk attachment with external storage

As external storage is not associated with a node, we have to make an
exception for that before raising an error.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoAdd more documentation to testutils_ssh.py
Helga Velroyen [Tue, 1 Dec 2015 15:20:57 +0000 (16:20 +0100)]
Add more documentation to testutils_ssh.py

This patch adds more comments to the functions in
testutils_ssh.py, in particular to clarify which function
returns what types of objects.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agorenew-crypto: use bulk-removal of SSH keys
Helga Velroyen [Tue, 24 Nov 2015 12:01:46 +0000 (13:01 +0100)]
renew-crypto: use bulk-removal of SSH keys

This patch makes renew-crypto use the newly introduced
bulk-removal function for SSH keys. This way the
complexity of renew-crypto (in terms of number of
SSH connections) becomes linear (from previously
quadratic).

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoUse bulk-removal of SSH keys for single keys
Helga Velroyen [Tue, 24 Nov 2015 10:33:29 +0000 (11:33 +0100)]
Use bulk-removal of SSH keys for single keys

As the code for bulk-removal of SSH keys subsumes
the code for removing a single SSH key, let the
latter call the first.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoBulk-removing SSH keys of diverse set of nodes
Helga Velroyen [Fri, 20 Nov 2015 10:16:58 +0000 (11:16 +0100)]
Bulk-removing SSH keys of diverse set of nodes

This patch adds a unit test where SSH keys of a diverse
set of nodes is removed. By 'diverse', we mean a set
consisting of master candidates, potential master
candidates, and normal nodes.

It also fixes some minor bug that surfaced with that
test.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoBulk-removal of SSH keys of normal nodes
Helga Velroyen [Fri, 20 Nov 2015 09:41:12 +0000 (10:41 +0100)]
Bulk-removal of SSH keys of normal nodes

This patch adds a unit test for bulk-removing
normal nodes. Besides that, it fixes a small
bug that surfaced with that test.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoBulk-remove SSH keys of potential master candidates
Helga Velroyen [Fri, 20 Nov 2015 09:30:08 +0000 (10:30 +0100)]
Bulk-remove SSH keys of potential master candidates

This patch adds a unit test for bulk-removing potential
master candidates.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoBulk-removal of SSH keys
Helga Velroyen [Fri, 20 Nov 2015 09:11:44 +0000 (10:11 +0100)]
Bulk-removal of SSH keys

In order to improve the runtime complexity of
'renew-crypto', this patch adds a function to
bulk-remove SSH keys of nodes (in contrast to
the function that only removes one key at a time).

Within this patch, it is only called in a unit
test. Further patches will integrate and test it
further.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agotestutils: add keys to own 'authorized_keys' file
Helga Velroyen [Tue, 24 Nov 2015 10:11:41 +0000 (11:11 +0100)]
testutils: add keys to own 'authorized_keys' file

This patch updates the SSH testutils to match reality better.
So far, the test framework did not consider the fact that
the key of each node should be added to it's own
'authorized_keys' file, even if the node is not a master
candidate. This patch fixes that to represent the production
behavior more accurately.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMake mock SSH file manager deal with lists
Helga Velroyen [Thu, 19 Nov 2015 15:13:17 +0000 (16:13 +0100)]
Make mock SSH file manager deal with lists

There was a subtle bug in the unit test of backend.py
which was masking another subtle bug in the test framework
in testutils_ssh.py.

As relict from some previous refactoring, the ssh.py
functions assume that there can be more than one public
key per node. The testutils so far assume there is only
one key per node and due to a bug, this cancelled out
nicely and was not found so far.

As we actually only have one key per node, the elegant
thing to do would be to adapt ssh.py rather than the
testutils, but that will break the interface of the
ssh_update.py tool. Since we would rather not do that
in a stable, branch, this patch adapts the testutils.
The adaption of the ssh.py will be done in a newer
branch then.

Additionally, this patch also sprinkles assertions
everywhere to ensure finding these kind of type messups
sooner.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoDon't deepcopy the config if the old value is not needed
Klaus Aehlig [Mon, 14 Dec 2015 14:08:22 +0000 (15:08 +0100)]
Don't deepcopy the config if the old value is not needed

The _UpgradeConfig function carries out internal upgrades of the
configuration, and additionally, if requested, saves the configuration
in case it changed in this process. To compare the old and the new
version, a deep copy of the old version is kept. As deep copying large
configurations is an expensive operation, only do it, if the value is
used afterwards.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoRevision bump for 2.15.2 v2.15.2
Hrvoje Ribicic [Wed, 16 Dec 2015 12:16:57 +0000 (12:16 +0000)]
Revision bump for 2.15.2

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUpdate NEWS file for 2.15.2
Hrvoje Ribicic [Wed, 16 Dec 2015 12:16:39 +0000 (12:16 +0000)]
Update NEWS file for 2.15.2

With the security information and a list of minor changes.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMerge branch 'stable-2.14' into stable-2.15
Hrvoje Ribicic [Wed, 16 Dec 2015 11:09:38 +0000 (12:09 +0100)]
Merge branch 'stable-2.14' into stable-2.15

* stable-2.14
  Revision bump for 2.14.2
  Update NEWS file for 2.14.2

* stable-2.13
  Revision bump for 2.13.3
  Update NEWS file for 2.13.3

* stable-2.12
  Bump revision number for 2.12.6
  Update NEWS file for 2.12.6

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Merge entries
  configure.ac - Take 2.15 revision numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoRevision bump for 2.14.2 v2.14.2
Hrvoje Ribicic [Tue, 15 Dec 2015 17:54:17 +0000 (18:54 +0100)]
Revision bump for 2.14.2

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUpdate NEWS file for 2.14.2
Hrvoje Ribicic [Tue, 15 Dec 2015 17:53:11 +0000 (18:53 +0100)]
Update NEWS file for 2.14.2

With the security issues text and a list of minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMerge branch 'stable-2.13' into stable-2.14
Hrvoje Ribicic [Tue, 15 Dec 2015 14:44:16 +0000 (15:44 +0100)]
Merge branch 'stable-2.13' into stable-2.14

* stable-2.13
  Revision bump for 2.13.3
  Update NEWS file for 2.13.3

* stable-2.12
  Bump revision number for 2.12.6
  Update NEWS file for 2.12.6

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Merged entries
  configure.ac - Took 2.14 version numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoRevision bump for 2.13.3 v2.13.3
Hrvoje Ribicic [Mon, 14 Dec 2015 18:00:43 +0000 (19:00 +0100)]
Revision bump for 2.13.3

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUpdate NEWS file for 2.13.3
Hrvoje Ribicic [Mon, 14 Dec 2015 17:59:26 +0000 (18:59 +0100)]
Update NEWS file for 2.13.3

With the security issues text and a list of minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMerge branch 'stable-2.12' into stable-2.13
Hrvoje Ribicic [Mon, 14 Dec 2015 17:33:14 +0000 (18:33 +0100)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Bump revision number for 2.12.6
  Update NEWS file for 2.12.6

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Merge entries
  configure.ac - Take 2.13 version numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoBump revision number for 2.12.6 v2.12.6
Hrvoje Ribicic [Mon, 14 Dec 2015 16:42:03 +0000 (17:42 +0100)]
Bump revision number for 2.12.6

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoUpdate NEWS file for 2.12.6
Hrvoje Ribicic [Mon, 14 Dec 2015 16:41:09 +0000 (17:41 +0100)]
Update NEWS file for 2.12.6

With the security issues text and a list of minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.11' into stable-2.12
Hrvoje Ribicic [Mon, 14 Dec 2015 16:15:14 +0000 (17:15 +0100)]
Merge branch 'stable-2.11' into stable-2.12

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Merged entries
  configure.ac - Took 2.12 version numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoRevision bump for 2.11.8 v2.11.8
Hrvoje Ribicic [Mon, 14 Dec 2015 14:07:23 +0000 (15:07 +0100)]
Revision bump for 2.11.8

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoUpdate NEWS file for 2.11.8
Hrvoje Ribicic [Mon, 14 Dec 2015 14:06:50 +0000 (15:06 +0100)]
Update NEWS file for 2.11.8

With the security issues text and a list of minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoFix error message in attachInstanceDiskChecks
Lisa Velden [Mon, 14 Dec 2015 14:13:10 +0000 (15:13 +0100)]
Fix error message in attachInstanceDiskChecks

Name the instance where disks are already attached to, which is not
necessarily the instance where we want to attach a disk to.
This fixes issue 1151.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMerge branch 'stable-2.10' into stable-2.11
Hrvoje Ribicic [Mon, 14 Dec 2015 13:13:03 +0000 (14:13 +0100)]
Merge branch 'stable-2.10' into stable-2.11

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Combine NEWS entries from both versions
  configure.ac - Take correct version numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoVersion bump for 2.10.8 v2.10.8
Hrvoje Ribicic [Fri, 11 Dec 2015 11:09:21 +0000 (12:09 +0100)]
Version bump for 2.10.8

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoUpdate NEWS file for 2.10.8
Hrvoje Ribicic [Fri, 11 Dec 2015 11:08:22 +0000 (12:08 +0100)]
Update NEWS file for 2.10.8

With the security issues text and list minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.9' into stable-2.10
Hrvoje Ribicic [Thu, 10 Dec 2015 18:04:48 +0000 (19:04 +0100)]
Merge branch 'stable-2.9' into stable-2.10

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - leave 2.9.7 info in
  configure.ac - revert version bump

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoBump revision number stable-2.9 v2.9.7
Hrvoje Ribicic [Thu, 10 Dec 2015 16:40:51 +0000 (17:40 +0100)]
Bump revision number

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUpdate NEWS file for 2.9.7 release
Hrvoje Ribicic [Thu, 10 Dec 2015 16:39:53 +0000 (17:39 +0100)]
Update NEWS file for 2.9.7 release

... with security release info and minor changes.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoImprove RAPI section on security
Hrvoje Ribicic [Thu, 10 Dec 2015 13:22:01 +0000 (14:22 +0100)]
Improve RAPI section on security

The RAPI section on security has been improved with new information
related on how users can lock RAPI down as they see fit, and what are
the risks involved with default settings.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoUpdate documentation of harep
Klaus Aehlig [Thu, 3 Dec 2015 10:24:31 +0000 (11:24 +0100)]
Update documentation of harep

Be more explicit about which action is taken by harep under
which conditions. In particular, mention the limitation that
harep never carries out migration operations.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoDocument harep --dry-run in the man page
Klaus Aehlig [Thu, 3 Dec 2015 08:52:32 +0000 (09:52 +0100)]
Document harep --dry-run in the man page

Document the new --dry-run option in harep's man page.
Also mention the limitations, as harep records its state
in instance tags.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoSupport --dry-run in harep
Klaus Aehlig [Wed, 2 Dec 2015 16:20:46 +0000 (17:20 +0100)]
Support --dry-run in harep

Add a --dry-run option to harep, so that users can verify
that the actions taken by harep are the ones they want.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoAdd a --dry-run option to htools
Klaus Aehlig [Wed, 2 Dec 2015 13:51:56 +0000 (14:51 +0100)]
Add a --dry-run option to htools

Add a new flag, --dry-run, to the available flags in htools.
It will be used for harep to allow diagnose-only runs.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.14' into stable-2.15
Hrvoje Ribicic [Fri, 4 Dec 2015 15:06:50 +0000 (16:06 +0100)]
Merge branch 'stable-2.14' into stable-2.15

* stable-2.14
  Fix lines with more than 80 characters
  Add more detach/attach sequence tests
  Allow disk attachment to diskless instances
  Improve tests for attaching disks

* stable-2.13
  (no changes)

* stable-2.12
  Restrict showing of DRBD secret using types
  Calculate correct affected nodes set in InstanceChangeGroup

* stable-2.11
  (no changes)

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.13' into stable-2.14
Hrvoje Ribicic [Thu, 3 Dec 2015 22:55:20 +0000 (22:55 +0000)]
Merge branch 'stable-2.13' into stable-2.14

* stable-2.13
  (no changes)

* stable-2.12
  Restrict showing of DRBD secret using types
  Calculate correct affected nodes set in InstanceChangeGroup

* stable-2.11
  (no changes)

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  src/Ganeti/Objects.hs - Followed code to Disk.hs
  test/hs/Test/Ganeti/Objects.hs - Added Private to disk definition

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoMerge branch 'stable-2.12' into stable-2.13
Hrvoje Ribicic [Thu, 3 Dec 2015 21:13:39 +0000 (21:13 +0000)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Restrict showing of DRBD secret using types
  Calculate correct affected nodes set in InstanceChangeGroup

* stable-2.11
  (no changes)

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoRestrict showing of DRBD secret using types
Hrvoje Ribicic [Tue, 1 Dec 2015 16:11:38 +0000 (16:11 +0000)]
Restrict showing of DRBD secret using types

While the Python changes from 2.9 do prevent Ganeti from accidentally
revealing the Haskell secret, they may not do so forever. The queries
are planned to switch from Python to Haskell at some point, and should
someone want to use the DRBD secret, they can do so easily.

As a more elegant way of hiding the secret, wrap it in a Private
wrapper, preventing it from leaking out unless explicitly requested.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.11' into stable-2.12
Hrvoje Ribicic [Tue, 1 Dec 2015 15:57:49 +0000 (15:57 +0000)]
Merge branch 'stable-2.11' into stable-2.12

* stable-2.11
  (no changes)

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  lib/client/gnt_instance.py - taken the 2.11 version, with explicit
                               parameter use
  qa/qa_rapi.py - merged imports, resolved trivial conflict

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.10' into stable-2.11
Hrvoje Ribicic [Mon, 30 Nov 2015 16:12:42 +0000 (17:12 +0100)]
Merge branch 'stable-2.10' into stable-2.11

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  qa/qa_rapi.py - simply append new changes

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.9' into stable-2.10
Hrvoje Ribicic [Mon, 30 Nov 2015 15:49:09 +0000 (16:49 +0100)]
Merge branch 'stable-2.9' into stable-2.10

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  lib/cmdlib/instance_query.py - removed physical_id changes

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoQA: Ensure the DRBD secret is not retrievable via RAPI
Hrvoje Ribicic [Fri, 27 Nov 2015 17:32:42 +0000 (17:32 +0000)]
QA: Ensure the DRBD secret is not retrievable via RAPI

The best way to ensure that the DRBD secret does not inadvertently leak
is to introduce a QA test examining the output of the interface in
which the leak was originally introduced.

The test added determines the DRBD secret and makes RAPI requests,
examining them for its presence and failing if a match is found.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoRedact the DRBD secret in instance queries
Hrvoje Ribicic [Fri, 27 Nov 2015 15:58:13 +0000 (15:58 +0000)]
Redact the DRBD secret in instance queries

As the DRBD secret should be used only by Ganeti internals, replacing
the actual secret with None does not hamper Ganeti's work, while
preventing the secret from being leaked.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoDo not attempt to use the DRBD secret in gnt-instance info
Hrvoje Ribicic [Fri, 21 Aug 2015 19:46:18 +0000 (19:46 +0000)]
Do not attempt to use the DRBD secret in gnt-instance info

... so just redact what is output.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoFix lines with more than 80 characters
Lisa Velden [Fri, 27 Nov 2015 10:25:55 +0000 (11:25 +0100)]
Fix lines with more than 80 characters

Previous refactoring has introduced lines with too many characters.
This patch fixes this.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoFix lines with more than 80 characters
Lisa Velden [Fri, 27 Nov 2015 10:25:55 +0000 (11:25 +0100)]
Fix lines with more than 80 characters

Previous refactoring has introduced lines with too many characters.
This patch fixes this.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoAdd more detach/attach sequence tests
Lisa Velden [Wed, 25 Nov 2015 16:57:18 +0000 (17:57 +0100)]
Add more detach/attach sequence tests

Test detach/attach sequences with an instance that becomes diskless
after detaching its disk and also test detach/attach with drbd disks.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoAllow disk attachment to diskless instances
Lisa Velden [Wed, 25 Nov 2015 15:00:45 +0000 (16:00 +0100)]
Allow disk attachment to diskless instances

As only DRBD disks can be associated to more nodes than the instance
where we want to attach the disk to, we have to change the check for
associated nodes, too.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoImprove tests for attaching disks
Lisa Velden [Wed, 25 Nov 2015 13:53:39 +0000 (14:53 +0100)]
Improve tests for attaching disks

by associating disks and instances to a specific node.
Also refactor mock uuids and mock disk names into variables.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoAdd more detach/attach sequence tests
Lisa Velden [Wed, 25 Nov 2015 16:57:18 +0000 (17:57 +0100)]
Add more detach/attach sequence tests

Test detach/attach sequences with an instance that becomes diskless
after detaching its disk and also test detach/attach with drbd disks.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoAllow disk attachment to diskless instances
Lisa Velden [Wed, 25 Nov 2015 15:00:45 +0000 (16:00 +0100)]
Allow disk attachment to diskless instances

As only DRBD disks can be associated to more nodes than the instance
where we want to attach the disk to, we have to change the check for
associated nodes, too.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoImprove tests for attaching disks
Lisa Velden [Wed, 25 Nov 2015 13:53:39 +0000 (14:53 +0100)]
Improve tests for attaching disks

by associating disks and instances to a specific node.
Also refactor mock uuids and mock disk names into variables.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoCompute lock allocation strictly
Klaus Aehlig [Thu, 26 Nov 2015 16:49:38 +0000 (17:49 +0100)]
Compute lock allocation strictly

Given that on updates it has to be fully computed anyway, do not
accumulate thunks during the computation.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUse only string value in error message
Lisa Velden [Mon, 23 Nov 2015 14:42:09 +0000 (15:42 +0100)]
Use only string value in error message

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoCalculate correct affected nodes set in InstanceChangeGroup
Oleg Ponomarev [Fri, 20 Nov 2015 20:45:11 +0000 (21:45 +0100)]
Calculate correct affected nodes set in InstanceChangeGroup

This is the fix for the issue 1144. The nodes affected by the
InstanceChangeGroup logical unit were calculated incorrectly and that
broke 'gnt-instance change-group --to' operation. This patch fixes it.

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.15' into stable-2.16
Helga Velroyen [Fri, 20 Nov 2015 10:34:44 +0000 (11:34 +0100)]
Merge branch 'stable-2.15' into stable-2.16

* stable-2.15
  Document the decission why optimisation is turned off
  Don't keep input for error messages
  Use dict.copy instead of deepcopy
  Use bulk-adding of keys in renew-crypto
  Make NodeSshKeyAdd use its *Bulk companion
  Unit test bulk-adding normal nodes
  Unit test for bulk-adding pot. master candidates
  Introduce bulk-adding of SSH keys
  Pause watcher during performance QA
  Send answers strictly
  Store keys as ByteStrings
  Encode UUIDs as ByteStrings
  Prefer the UuidObject type class over specific functions
  Assign the variables before use (bugfix for dee6adb9)
  Extend QA to detect autopromotion errors
  Handle SSH key distribution on auto promotion
  Do not remove authorized key of node itself
  Fix indentation
  Support force option for deactivate disks on RAPI

* stable-2.14
  Fix faulty iallocator type check
  Improve cfgupgrade output in case of errors

* stable-2.13
  Extend timeout for gnt-cluster renew-crypto
  Reduce flakyness of GetCmdline test on slow machines
  Remove duplicated words

* stable-2.12
  Revert "Also consider connection time out a network error"
  Clone lists before modifying
  Make lockConfig call retryable
  Return the correct error code in the post-upgrade script
  Make openssl refrain from DH altogether
  Fix upgrades of instances with missing creation time

* stable-2.11
  (none)

* stable-2.10
  Remove -X from hspace man page
  Make htools tolerate missing "dtotal" and "dfree" on luxi

Conflicts:
  lib/backend.py
  lib/cmdlib/node.py
  src/Ganeti/WConfd/ConfigModifications.hs

Resolutions:
  lib/backend.py
    use bulk-adding keys with renamed public key file variable
  lib/cmdlib/node.py
    use self.cfg.RemoveNode rather than self.context.RemoveNode
  src/Ganeti/WConfd/ConfigModifications.hs
    fix imports
    add UTF8.{to,from}String at appropriate places

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoAdd entries describing new gnt-cluster params to manpage
Hrvoje Ribicic [Mon, 9 Nov 2015 17:49:53 +0000 (18:49 +0100)]
Add entries describing new gnt-cluster params to manpage

And also sprinkle reminders of when to update them across the codebase.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoQA: Add ssh-key-type and -bits tests
Hrvoje Ribicic [Mon, 9 Nov 2015 17:18:33 +0000 (18:18 +0100)]
QA: Add ssh-key-type and -bits tests

This patch expands the testing of SSH key renewal by changing the key
type existing on a cluster during the QA.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoQA: Extend AssertCommand to allow not forwarding the agent
Hrvoje Ribicic [Fri, 6 Nov 2015 16:01:42 +0000 (16:01 +0000)]
QA: Extend AssertCommand to allow not forwarding the agent

When testing SSH-related behavior in Ganeti, having the SSH agent
forwarded in all the command-running utilities can produce spurious
errors, or worse yet, allow real ones to sneak by. In this patch, the
AssertCommand function is extended to allow disabling of agent
forwarding. This also switches off connection multiplexing, as the
multiplexed connection forwards agents implicitly.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoRemove default limit on diffs in cfgupgrade tests
Hrvoje Ribicic [Fri, 6 Nov 2015 12:48:01 +0000 (12:48 +0000)]
Remove default limit on diffs in cfgupgrade tests

These tests deal with large configuration files, and without the
changes present in this patch, instead of a pretty git-style diff of
two configurations, we get nothing.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoQA: Downgrade the cluster key type in 2.16
Hrvoje Ribicic [Fri, 6 Nov 2015 01:53:50 +0000 (02:53 +0100)]
QA: Downgrade the cluster key type in 2.16

The downgrade/upgrade QA test starts from a freshly-built cluster which
would have RSA keys in 2.16. Downgrading such a cluster is prevented by
one of the preceding patches, for good reason, so this patch makes sure
to switch to DSA keys before running the upgrade test.

As this code is meant to be here only in 2.16, it also includes a kill
switch in case it is merged up silently.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoFix typo
Hrvoje Ribicic [Fri, 6 Nov 2015 01:53:00 +0000 (02:53 +0100)]
Fix typo

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoFail early for invalid key type and size combinations
Hrvoje Ribicic [Fri, 6 Nov 2015 01:35:51 +0000 (02:35 +0100)]
Fail early for invalid key type and size combinations

The ssh-keygen utility permits only some combinations of key types and
bit sizes. As many more things can go wrong late in the renewal
process, this patch introduces prerequisite checks mimicking those of
ssh-keygen.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoHandle SSH key changes in upgrades and downgrades
Hrvoje Ribicic [Thu, 5 Nov 2015 13:13:58 +0000 (14:13 +0100)]
Handle SSH key changes in upgrades and downgrades

When performing an upgrade of an old cluster, it is necessary to set
the SSH key parameters to the exact same values earlier versions
implicitly used - DSA with 1024 bits.

In the other direction, we simply do not permit downgrades if keys
other than DSA are being used. Triggering a gnt-cluster renew-crypto
might be time-consuming and surprising for the user, so we are simply
throwing out an error message, explaining that the downgrade cannot be
performed in the current state of the cluster.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>