ganeti-github.git
4 years agoBump version suffix to 2.16.0 rc1 v2.16.0rc1
Viktor Bachraty [Wed, 17 Feb 2016 13:47:53 +0000 (13:47 +0000)]
Bump version suffix to 2.16.0 rc1

Signed-off-by: Viktor Bachraty <vbachraty@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoUpdate NEWS file for 2.16.0 rc1 release
Viktor Bachraty [Wed, 17 Feb 2016 13:47:55 +0000 (13:47 +0000)]
Update NEWS file for 2.16.0 rc1 release

Added entry to changes since beta2.

Signed-off-by: Viktor Bachraty <vbachraty@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoOn group verify, only flush to group nodes
Klaus Aehlig [Mon, 8 Feb 2016 10:24:46 +0000 (11:24 +0100)]
On group verify, only flush to group nodes

As group verification also verifies that all nodes
in this group have the same understanding of the configuration,
it needs to be flushed at the beginning of the verification.
However, it is enough to flush only to the nodes of that group.
So do this.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Brian Foley <bpfoley@google.com>

4 years agoSupport flushing to a single group
Klaus Aehlig [Mon, 8 Feb 2016 15:28:25 +0000 (16:28 +0100)]
Support flushing to a single group

When a group is verified, the configuration needs to be
fully flushed to that group. Support flushing by group,
so that we can get rid of the over-approximation of flushing
everything.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Brian Foley <bpfoley@google.com>

4 years agoMake config distribution parametric in target group
Klaus Aehlig [Mon, 8 Feb 2016 15:00:45 +0000 (16:00 +0100)]
Make config distribution parametric in target group

Sometimes, it is sufficient to force distribution of the configuration
only to a single target group. Hence make the infrastructure parametric
in the groups the configuration should be distributed to.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Brian Foley <bpfoley@google.com>

4 years agoUpdate NEWS file for 2.16.0 beta2 v2.16.0beta2
Viktor Bachraty [Tue, 2 Feb 2016 14:22:02 +0000 (14:22 +0000)]
Update NEWS file for 2.16.0 beta2

Update minor changes with fixes that happened after the originally
proposed release date. Update actual date of release.

Signed-off-by: Viktor Bachraty <vbachraty@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.15' into stable-2.16
Klaus Aehlig [Mon, 1 Feb 2016 11:59:37 +0000 (12:59 +0100)]
Merge branch 'stable-2.15' into stable-2.16

* stable-2.15
  Do not add a new Inotify watchers on timer
  Mock InitDrbdHelper's output in unittests
  Optimise codegen for Python OpCode classes

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoDo not add a new Inotify watchers on timer
Klaus Aehlig [Thu, 28 Jan 2016 18:13:00 +0000 (19:13 +0100)]
Do not add a new Inotify watchers on timer

Ganeti updates its in-memory copy of the configuration in several ways.
One of them is by using an inotify, the other is by periodically, in the
order of seconds, polling the file. On the latter, the inotify does not
have to be reinstantiated; in fact, doing so will result in actions taken
several times, once the inotify actually fires. Fix the fact, it was
reinstantiated.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoMock InitDrbdHelper's output in unittests
Helga Velroyen [Thu, 28 Jan 2016 17:56:45 +0000 (18:56 +0100)]
Mock InitDrbdHelper's output in unittests

The output of the InitDrbdHelper function was cluttering up
the unit tests. Let's mock that output in tests.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoOptimise codegen for Python OpCode classes
Brian Foley [Wed, 27 Jan 2016 19:23:23 +0000 (19:23 +0000)]
Optimise codegen for Python OpCode classes

This makes hs2py output types like ht.TMaybeBool instead of
ht.TMaybe(ht.TBool). The two have equivalent behaviour, but Python
creates a new callable object at runtime for each instance of the
second, because TMaybe is a higher order function.

This optimisation saves >500kB of heap for "import opcodes" for every
Ganeti Python process.

Signed-off-by: Brian Foley <bpfoley@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

4 years agoUpdate NEWS file for 2.16.0 beta2
Viktor Bachraty [Wed, 27 Jan 2016 20:38:13 +0000 (20:38 +0000)]
Update NEWS file for 2.16.0 beta2

Update both major and minor changes since beta 1 including changes
inherited from older branches.

Signed-off-by: Viktor Bachraty <vbachraty@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoBump version suffix to 2.16.0 beta2
Viktor Bachraty [Wed, 27 Jan 2016 20:50:23 +0000 (20:50 +0000)]
Bump version suffix to 2.16.0 beta2

Signed-off-by: Viktor Bachraty <vbachraty@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoMerge branch 'stable-2.15' into stable-2.16
Hrvoje Ribicic [Fri, 22 Jan 2016 12:52:50 +0000 (13:52 +0100)]
Merge branch 'stable-2.15' into stable-2.16

* stable-2.15
  (no changes)

* stable-2.14
  Fix failover in case the source node is offline

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoMerge branch 'stable-2.14' into stable-2.15
Hrvoje Ribicic [Fri, 22 Jan 2016 11:26:07 +0000 (12:26 +0100)]
Merge branch 'stable-2.14' into stable-2.15

* stable-2.14
  Fix failover in case the source node is offline

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoSet block buffering for UDSServer
Klaus Aehlig [Thu, 21 Jan 2016 14:45:23 +0000 (15:45 +0100)]
Set block buffering for UDSServer

Commit b0a7e3771bfd changed sending of JSON-encoded answers
to standard String sending. This was necessary as converting
Strings to ByteStrings, even to lazy ones, fully enforced the
String before the first Char got out of scope and could be
garbage collected.  The down-side of this approach is, that
we now end up with one system call per character to be send.
The good news, however, is that the library's buffering uses
memory only a little more than a byte for a byte, so we can
afford buffering in that layer. Do so to reduce the number of
system calls.

On a, not quite realistic, test cluster, this resulted in the
time for a config-read going down by 1.5 orders of magnitude
with only small increase in residual memory.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoFix failover in case the source node is offline
Dimitris Aragiorgis [Wed, 20 Jan 2016 16:37:14 +0000 (18:37 +0200)]
Fix failover in case the source node is offline

Commit ff74b60 closes instance disks on the source node before
doing a failover. In case the node is offline this is not possible.
This patch proceeds with the failover in case the source node
is offline or the --ingore-consistency flag is set. Reduce also
some config lookups for the node's name.

This fixes Issue #1162.

Signed-off-by: Dimitris Aragiorgis <dimara@arrikto.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoDocument the increased timeout as an incompatible change
Klaus Aehlig [Wed, 20 Jan 2016 11:13:33 +0000 (12:13 +0100)]
Document the increased timeout as an incompatible change

While the timeout for communication with luxid is mainly
an internal parameter, it also changes which response time
for Ganeti tools is still to be considered normal. Hence
warn users that might have higher level tools interacting
with Ganeti.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoIncrease timeouts for luxi by a factor of 3
Klaus Aehlig [Wed, 20 Jan 2016 11:07:03 +0000 (12:07 +0100)]
Increase timeouts for luxi by a factor of 3

While sending answers lazily as Strings has reduced memory footprint
by over an order of magnitude, it seems that answer times have gotten
slower. Accept this trade off treating time for space and increase all
timeouts accordingly.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoDo not repeat constants in comments
Klaus Aehlig [Wed, 20 Jan 2016 11:02:49 +0000 (12:02 +0100)]
Do not repeat constants in comments

...as this works against the idea of having all constants in one
central place so that they can be changed in a simple way.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoMerge branch 'stable-2.15' into stable-2.16
Klaus Aehlig [Fri, 15 Jan 2016 13:59:47 +0000 (14:59 +0100)]
Merge branch 'stable-2.15' into stable-2.16

* stable-2.15
  Catch IOError of SSH files when removing node
  Fix renew-crypto on one-node-cluster
  ssh_update: log data that is received
  Increase timeout of RPC adding/removing keys
  After TestNodeModify, fix the pool of master candidates

* stable-2.14
  Test disk attachment with different primary nodes
  Check for same primary node before disk attachment
  Add detach/attach sequence test
  Allow disk attachment with external storage

* stable-2.13
  Run ssh-key renewal in debug mode during upgrade

* stable-2.12
  Increase minimal sizes of test online nodes
  Also log the high-level upgrade steps
  Add function to provide logged user feedback
  Run renew-crypto in upgrades in debug mode
  Unconditionally log upgrades at debug level
  Document healthy-majority restriction on master-failover
  Check for healthy majority on master failover with voting
  Add a predicate testing that a majority of nodes is healthy
  Fix outdated comment
  Pass arguments to correct daemons during master-failover
  Fix documentation for master-failover

* stable-2.11
  (no changes)

* stable-2.10
  KVM: explicitly configure routed NICs late

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.14' into stable-2.15
Klaus Aehlig [Fri, 15 Jan 2016 10:17:06 +0000 (11:17 +0100)]
Merge branch 'stable-2.14' into stable-2.15

* stable-2.14
  Test disk attachment with different primary nodes
  Check for same primary node before disk attachment
  Add detach/attach sequence test
  Allow disk attachment with external storage

* stable-2.13
  Run ssh-key renewal in debug mode during upgrade

* stable-2.12
  Increase minimal sizes of test online nodes
  Also log the high-level upgrade steps
  Add function to provide logged user feedback
  Run renew-crypto in upgrades in debug mode
  Unconditionally log upgrades at debug level
  Document healthy-majority restriction on master-failover
  Check for healthy majority on master failover with voting
  Add a predicate testing that a majority of nodes is healthy
  Fix outdated comment
  Pass arguments to correct daemons during master-failover
  Fix documentation for master-failover

* stable-2.11
  (no changes)

* stable-2.10
  KVM: explicitly configure routed NICs late

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.13' into stable-2.14
Klaus Aehlig [Thu, 14 Jan 2016 17:01:17 +0000 (18:01 +0100)]
Merge branch 'stable-2.13' into stable-2.14

* stable-2.13
  Run ssh-key renewal in debug mode during upgrade

* stable-2.12
  Increase minimal sizes of test online nodes
  Also log the high-level upgrade steps
  Add function to provide logged user feedback
  Run renew-crypto in upgrades in debug mode
  Unconditionally log upgrades at debug level
  Document healthy-majority restriction on master-failover
  Check for healthy majority on master failover with voting
  Add a predicate testing that a majority of nodes is healthy
  Fix outdated comment
  Pass arguments to correct daemons during master-failover
  Fix documentation for master-failover

* stable-2.11
  (no changes)

* stable-2.10
  KVM: explicitly configure routed NICs late

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoRun ssh-key renewal in debug mode during upgrade
Klaus Aehlig [Thu, 14 Jan 2016 14:10:01 +0000 (15:10 +0100)]
Run ssh-key renewal in debug mode during upgrade

As errors during an upgrade of Ganeti are harder to
understand, as two versions of Ganeti are involved,
provide more debug information for everything that happens
during that process. Note that upgrades are a rare event,
so we do not have to worry about the size of log files
too much.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoMerge branch 'stable-2.12' into stable-2.13
Klaus Aehlig [Thu, 14 Jan 2016 13:07:02 +0000 (14:07 +0100)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Increase minimal sizes of test online nodes
  Also log the high-level upgrade steps
  Add function to provide logged user feedback
  Run renew-crypto in upgrades in debug mode
  Unconditionally log upgrades at debug level
  Document healthy-majority restriction on master-failover
  Check for healthy majority on master failover with voting
  Add a predicate testing that a majority of nodes is healthy
  Fix outdated comment
  Pass arguments to correct daemons during master-failover
  Fix documentation for master-failover

* stable-2.11
  (no changes)

* stable-2.10
  KVM: explicitly configure routed NICs late

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoIncrease minimal sizes of test online nodes stable-2.12
Klaus Aehlig [Tue, 8 Dec 2015 16:05:10 +0000 (17:05 +0100)]
Increase minimal sizes of test online nodes

A lot of our tests work by generating a node and a
strictly smaller instance and then continue under
the assumption that the instance will fit on the node.
To obtain a strictly smaller instance, we take an instance
of size at most half the free resources of the node. The
problem with this approach is that we also require minimal
resources of an instance (for examples to be realistic); now,
this can lead to an upper bound lower than the lower bound
and, by the way QuickCheck's `choose` works, still a value
between these bounds is chosen, violating the assumptions
about node and instance sizes.

To avoid those problems, set the minimal resources of an
allocatable node so that half of them is still bigger than
the minimal resources of an instance.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

Cherry-picked-from: 6ccf05c1507c58e
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoMerge branch 'stable-2.11' into stable-2.12
Klaus Aehlig [Tue, 12 Jan 2016 15:46:43 +0000 (16:46 +0100)]
Merge branch 'stable-2.11' into stable-2.12

* stable-2.11
  (no changes)

* stable-2.10
  KVM: explicitly configure routed NICs late

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoAlso log the high-level upgrade steps
Klaus Aehlig [Tue, 12 Jan 2016 10:37:13 +0000 (11:37 +0100)]
Also log the high-level upgrade steps

The upgrade of a Ganeti cluster is done in several
high-level steps ("Draining queue", "Pausing the watcher",
"Stopping daemons", ...). Log those headings as well in
order to simplify reading the log file; with these headings,
it is more easy to understand which goal is aimed for with
all the micro-step RunCmd log entries.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoAdd function to provide logged user feedback
Klaus Aehlig [Tue, 12 Jan 2016 14:54:33 +0000 (15:54 +0100)]
Add function to provide logged user feedback

Add a utility function that provides feedback to the
user on stdout that is additionally logged (at INFO level)
in the log file.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoRun renew-crypto in upgrades in debug mode
Klaus Aehlig [Mon, 11 Jan 2016 17:11:43 +0000 (18:11 +0100)]
Run renew-crypto in upgrades in debug mode

As errors during an upgrade of Ganeti are harder to
understand, as two versions of Ganeti are involved,
provide more debug information for everything that happens
during that process. Note that upgrades are a rare event,
so we do not have to worry about the size of log files
too much.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoUnconditionally log upgrades at debug level
Klaus Aehlig [Tue, 12 Jan 2016 09:38:50 +0000 (10:38 +0100)]
Unconditionally log upgrades at debug level

Cluster upgrades to a new minor version of Ganeti are a rare
operation (in fact, new minor versions are released only every
3 months). Therefore, we do not have to worry about increased
size of log files. However, upgrades of Ganeti are complicated
in the sense that, should something break during the upgrade, it
is not immediately obvious, in which Ganeti state is left in. Therefore,
always provide full log information on upgrades. Fixes issue 1137.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoSend messages as Strings
Klaus Aehlig [Mon, 30 Nov 2015 14:19:38 +0000 (15:19 +0100)]
Send messages as Strings

ByteStrings are a more compact representation of a sequence of octets
than are Strings. However, converting a String into a ByteString, even
a lazy one, looks at a huge number of characters before the first goes
out of scope; thus the String gets enforced effectively. As Strings,
as a list of unicode characters, have a quite memory-intense representation,
this loss of lazyness results in a memory spike that is quite significant,
at least for restricted environments like a Xen dom0, when sending the
whole Ganeti configuration.

Therefore, send messages as String over the wire to preserve lazyness.
This is sound, as our JSON representation is 7-bit clean, and hence
every character coincides with its UTF8 encoding. On a larger cluster,
this saved an order of magnitude in peak memory usage.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.10' into stable-2.11 stable-2.11
Klaus Aehlig [Mon, 11 Jan 2016 11:30:30 +0000 (12:30 +0100)]
Merge branch 'stable-2.10' into stable-2.11

* stable-2.10
  KVM: explicitly configure routed NICs late

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoTest disk attachment with different primary nodes
Lisa Velden [Mon, 11 Jan 2016 11:12:49 +0000 (12:12 +0100)]
Test disk attachment with different primary nodes

Test a detach/attach sequence with a DRBD disk and a different primary
node for the disk and the instance. This should raise an exception.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoCheck for same primary node before disk attachment
Lisa Velden [Mon, 11 Jan 2016 11:09:47 +0000 (12:09 +0100)]
Check for same primary node before disk attachment

Make sure a DRBD disk has the same primary node as the instance where it
will be attached to.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoDocument healthy-majority restriction on master-failover
Klaus Aehlig [Fri, 8 Jan 2016 13:51:20 +0000 (14:51 +0100)]
Document healthy-majority restriction on master-failover

The previous patch introduced a behavioral change for master-failover:
it is rejected unless a majority of nodes is healthy or the --no-voting
option is given. (While we in general do not change behavior on a stable
branch, rejecting an operation that can be retried with different command-line
options is better than breaking the cluster completely.) Document this.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoCheck for healthy majority on master failover with voting
Klaus Aehlig [Fri, 8 Jan 2016 10:37:17 +0000 (11:37 +0100)]
Check for healthy majority on master failover with voting

The normal procedure for a master failover is that, after telling
each node the new master, the daemons on the new master node are
started the standard way, i.e., with voting. This, however, requires
that a majority of nodes is still healthy; otherwise, the failover
will result in the daemons not starting and thus a broken cluster.
Therefore, reject master failovers with voting, unless we can verify
that a majority of nodes is still responding.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoAdd a predicate testing that a majority of nodes is healthy
Klaus Aehlig [Fri, 8 Jan 2016 11:26:57 +0000 (12:26 +0100)]
Add a predicate testing that a majority of nodes is healthy

For standard master failover (with voting), it is necessary
that the majority of nodes is still reachable and can answer
questions about which node is master. Add a predicate verifying
that this is still true.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoFix outdated comment
Klaus Aehlig [Fri, 8 Jan 2016 10:54:47 +0000 (11:54 +0100)]
Fix outdated comment

Commit 5e641d0a introduced also counting the vote of
the node itself. Adapt the parameter description
accordingly.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoCatch IOError of SSH files when removing node
Helga Velroyen [Thu, 17 Dec 2015 09:03:17 +0000 (10:03 +0100)]
Catch IOError of SSH files when removing node

This patch catches an IOError when a node is removed
from a cluster and the SSH files of the node are messed
up. Previously, this caused the removal to fail, which
is not exactly what you want when removing a messed
up node.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Cherry-picked-from: a856040abc755b4
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoFix renew-crypto on one-node-cluster
Helga Velroyen [Wed, 16 Dec 2015 10:03:23 +0000 (11:03 +0100)]
Fix renew-crypto on one-node-cluster

There was a bug which made 'gnt-cluster renew-crypto'
crash if it is a one-node cluster. This patch fixes
it by checking if there are any non-master nodes
to update at all.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Cherry-picked-from: 88ac338d88465cc0b
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agossh_update: log data that is received
Helga Velroyen [Tue, 15 Dec 2015 14:03:53 +0000 (15:03 +0100)]
ssh_update: log data that is received

Debugging ssh_update can be annoying, because the data
used as input is not dumped anywhere. This patch logs
makes sure it gets logged (at DEBUG level) when
ssh_update receives the data.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Cherry-picked-from: 5c370ec180
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoKVM: explicitly configure routed NICs late stable-2.10
Apollon Oikonomopoulos [Wed, 2 Dec 2015 12:35:42 +0000 (14:35 +0200)]
KVM: explicitly configure routed NICs late

Commit cc8a8ed7 outlined the reasons for configuring bridged NICs early
during live migration and routed NICs after migration has been finished.
Back then these were the only types of NICs available, however with the
introduction of OVS support this has changed.

Since OVS bridges are essentially bridges, the considerations outlined
in cc8a8ed7 still apply: in particular, we do not want to lose the
gratuitous ARP sent out by the KVM NICs, so we have to configure
the OVS interfaces early in the migration process as well.

Rather than explicitly configure bridged and OVS interfaces early, we
prefer to explicitly configure routed interfaces late, since this leads
to more compact code.

Signed-off-by: Apollon Oikonomopoulos <apoikos@gmail.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoIncrease timeout of RPC adding/removing keys
Helga Velroyen [Thu, 7 Jan 2016 13:27:29 +0000 (14:27 +0100)]
Increase timeout of RPC adding/removing keys

This patch increases the timeout for the RPC calls that
add and remove SSH keys to the cluster. This is necessary,
because in big clusters the distribution/removal of a
key takes too long as Ganeti has to contact every node in
the cluster.

This patch increases the timeout from URGENT to FAST
(the next higher option).

The alternatives to this include splitting up the
RPC call to several calls, which will add addiional
overall runtime and RPC overhead as well as security
implications. Since the higher timeout was tested
in a big cluster, we go with this for now.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoAfter TestNodeModify, fix the pool of master candidates
Klaus Aehlig [Tue, 22 Dec 2015 11:35:40 +0000 (12:35 +0100)]
After TestNodeModify, fix the pool of master candidates

The test TestNodeModify temporarily modifies the cluster parameter
candidate-pool-size, which controls the minimal desirable number of
master candidates. Depending on the size of the test cluster, this
temporary modification can be a decrease (for clusters with up to 10
nodes) or an increase (for clusters with 12 or more nodes). Ganeti's
behavior upon change of the candidate pool size is to promote nodes to
master candidates upon increase, but do nothing upon decrease. This is
a safe behavior, as too many master candidates is not a problem; the
chance of data loss is even smaller. However, it means that the test
has a size effect of, for large test cluster, increasing the actual
number of nodes that are master candidates. While not a problem for
correctness, this side effect does affect our performance tests (which
usually are run after the functional tests) as more master candidates
means more nodes to replicate information to.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoPass arguments to correct daemons during master-failover
Hrvoje Ribicic [Thu, 17 Dec 2015 00:18:50 +0000 (00:18 +0000)]
Pass arguments to correct daemons during master-failover

A master-failover can be executed with the --no-voting flag, making
Ganeti start daemons despite a lack of votes. This is necessary to
fail over a cluster reduced to two nodes. The feature has not
been working since 2.12 daemon refactoring, as the daemon parameters
were passed through environmental variables that were not updated.

This commit passes the parameters correctly, and fixes issue 1159.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoFix typo 'option' instead of 'options'
Helga Velroyen [Tue, 5 Jan 2016 09:49:13 +0000 (10:49 +0100)]
Fix typo 'option' instead of 'options'

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoMerge branch 'stable-2.15' into stable-2.16
Helga Velroyen [Mon, 4 Jan 2016 16:07:50 +0000 (17:07 +0100)]
Merge branch 'stable-2.15' into stable-2.16

* stable-2.15
  Add more documentation to testutils_ssh.py
  renew-crypto: use bulk-removal of SSH keys
  Use bulk-removal of SSH keys for single keys
  Bulk-removing SSH keys of diverse set of nodes
  Bulk-removal of SSH keys of normal nodes
  Bulk-remove SSH keys of potential master candidates
  Bulk-removal of SSH keys
  testutils: add keys to own 'authorized_keys' file
  Make mock SSH file manager deal with lists
  Don't deepcopy the config if the old value is not needed
  Revision bump for 2.15.2
  Update NEWS file for 2.15.2
  Compute lock allocation strictly

* stable-2.14
  Revision bump for 2.14.2
  Update NEWS file for 2.14.2
  Fix lines with more than 80 characters
  Add more detach/attach sequence tests
  Allow disk attachment to diskless instances
  Improve tests for attaching disks

* stable-2.13
  Revision bump for 2.13.3
  Update NEWS file for 2.13.3

* stable-2.12
  Bump revision number for 2.12.6
  Update NEWS file for 2.12.6
  Restrict showing of DRBD secret using types
  Calculate correct affected nodes set in InstanceChangeGroup

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  NEWS
  configure.ac

Resolutions:
  NEWS: merge contents in right order
  configure.ac: keep version number of 2.16

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoFix documentation for master-failover
Hrvoje Ribicic [Mon, 4 Jan 2016 13:16:45 +0000 (14:16 +0100)]
Fix documentation for master-failover

The gnt-cluster manual still specified that arguments should be passed
to the master daemon - one which no longer exists. This patch specifies
the two new daemons to which arguments should be passed instead.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoAdd detach/attach sequence test
Lisa Velden [Wed, 16 Dec 2015 15:27:44 +0000 (16:27 +0100)]
Add detach/attach sequence test

Add a test for external storage.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoAllow disk attachment with external storage
Lisa Velden [Wed, 16 Dec 2015 13:57:43 +0000 (14:57 +0100)]
Allow disk attachment with external storage

As external storage is not associated with a node, we have to make an
exception for that before raising an error.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoAdd more documentation to testutils_ssh.py
Helga Velroyen [Tue, 1 Dec 2015 15:20:57 +0000 (16:20 +0100)]
Add more documentation to testutils_ssh.py

This patch adds more comments to the functions in
testutils_ssh.py, in particular to clarify which function
returns what types of objects.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agorenew-crypto: use bulk-removal of SSH keys
Helga Velroyen [Tue, 24 Nov 2015 12:01:46 +0000 (13:01 +0100)]
renew-crypto: use bulk-removal of SSH keys

This patch makes renew-crypto use the newly introduced
bulk-removal function for SSH keys. This way the
complexity of renew-crypto (in terms of number of
SSH connections) becomes linear (from previously
quadratic).

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoUse bulk-removal of SSH keys for single keys
Helga Velroyen [Tue, 24 Nov 2015 10:33:29 +0000 (11:33 +0100)]
Use bulk-removal of SSH keys for single keys

As the code for bulk-removal of SSH keys subsumes
the code for removing a single SSH key, let the
latter call the first.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoBulk-removing SSH keys of diverse set of nodes
Helga Velroyen [Fri, 20 Nov 2015 10:16:58 +0000 (11:16 +0100)]
Bulk-removing SSH keys of diverse set of nodes

This patch adds a unit test where SSH keys of a diverse
set of nodes is removed. By 'diverse', we mean a set
consisting of master candidates, potential master
candidates, and normal nodes.

It also fixes some minor bug that surfaced with that
test.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoBulk-removal of SSH keys of normal nodes
Helga Velroyen [Fri, 20 Nov 2015 09:41:12 +0000 (10:41 +0100)]
Bulk-removal of SSH keys of normal nodes

This patch adds a unit test for bulk-removing
normal nodes. Besides that, it fixes a small
bug that surfaced with that test.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoBulk-remove SSH keys of potential master candidates
Helga Velroyen [Fri, 20 Nov 2015 09:30:08 +0000 (10:30 +0100)]
Bulk-remove SSH keys of potential master candidates

This patch adds a unit test for bulk-removing potential
master candidates.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoBulk-removal of SSH keys
Helga Velroyen [Fri, 20 Nov 2015 09:11:44 +0000 (10:11 +0100)]
Bulk-removal of SSH keys

In order to improve the runtime complexity of
'renew-crypto', this patch adds a function to
bulk-remove SSH keys of nodes (in contrast to
the function that only removes one key at a time).

Within this patch, it is only called in a unit
test. Further patches will integrate and test it
further.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agotestutils: add keys to own 'authorized_keys' file
Helga Velroyen [Tue, 24 Nov 2015 10:11:41 +0000 (11:11 +0100)]
testutils: add keys to own 'authorized_keys' file

This patch updates the SSH testutils to match reality better.
So far, the test framework did not consider the fact that
the key of each node should be added to it's own
'authorized_keys' file, even if the node is not a master
candidate. This patch fixes that to represent the production
behavior more accurately.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMake mock SSH file manager deal with lists
Helga Velroyen [Thu, 19 Nov 2015 15:13:17 +0000 (16:13 +0100)]
Make mock SSH file manager deal with lists

There was a subtle bug in the unit test of backend.py
which was masking another subtle bug in the test framework
in testutils_ssh.py.

As relict from some previous refactoring, the ssh.py
functions assume that there can be more than one public
key per node. The testutils so far assume there is only
one key per node and due to a bug, this cancelled out
nicely and was not found so far.

As we actually only have one key per node, the elegant
thing to do would be to adapt ssh.py rather than the
testutils, but that will break the interface of the
ssh_update.py tool. Since we would rather not do that
in a stable, branch, this patch adapts the testutils.
The adaption of the ssh.py will be done in a newer
branch then.

Additionally, this patch also sprinkles assertions
everywhere to ensure finding these kind of type messups
sooner.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoDon't deepcopy the config if the old value is not needed
Klaus Aehlig [Mon, 14 Dec 2015 14:08:22 +0000 (15:08 +0100)]
Don't deepcopy the config if the old value is not needed

The _UpgradeConfig function carries out internal upgrades of the
configuration, and additionally, if requested, saves the configuration
in case it changed in this process. To compare the old and the new
version, a deep copy of the old version is kept. As deep copying large
configurations is an expensive operation, only do it, if the value is
used afterwards.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoRevision bump for 2.15.2 v2.15.2
Hrvoje Ribicic [Wed, 16 Dec 2015 12:16:57 +0000 (12:16 +0000)]
Revision bump for 2.15.2

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUpdate NEWS file for 2.15.2
Hrvoje Ribicic [Wed, 16 Dec 2015 12:16:39 +0000 (12:16 +0000)]
Update NEWS file for 2.15.2

With the security information and a list of minor changes.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMerge branch 'stable-2.14' into stable-2.15
Hrvoje Ribicic [Wed, 16 Dec 2015 11:09:38 +0000 (12:09 +0100)]
Merge branch 'stable-2.14' into stable-2.15

* stable-2.14
  Revision bump for 2.14.2
  Update NEWS file for 2.14.2

* stable-2.13
  Revision bump for 2.13.3
  Update NEWS file for 2.13.3

* stable-2.12
  Bump revision number for 2.12.6
  Update NEWS file for 2.12.6

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Merge entries
  configure.ac - Take 2.15 revision numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoRevision bump for 2.14.2 v2.14.2
Hrvoje Ribicic [Tue, 15 Dec 2015 17:54:17 +0000 (18:54 +0100)]
Revision bump for 2.14.2

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUpdate NEWS file for 2.14.2
Hrvoje Ribicic [Tue, 15 Dec 2015 17:53:11 +0000 (18:53 +0100)]
Update NEWS file for 2.14.2

With the security issues text and a list of minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMerge branch 'stable-2.13' into stable-2.14
Hrvoje Ribicic [Tue, 15 Dec 2015 14:44:16 +0000 (15:44 +0100)]
Merge branch 'stable-2.13' into stable-2.14

* stable-2.13
  Revision bump for 2.13.3
  Update NEWS file for 2.13.3

* stable-2.12
  Bump revision number for 2.12.6
  Update NEWS file for 2.12.6

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Merged entries
  configure.ac - Took 2.14 version numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoRevision bump for 2.13.3 v2.13.3
Hrvoje Ribicic [Mon, 14 Dec 2015 18:00:43 +0000 (19:00 +0100)]
Revision bump for 2.13.3

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUpdate NEWS file for 2.13.3
Hrvoje Ribicic [Mon, 14 Dec 2015 17:59:26 +0000 (18:59 +0100)]
Update NEWS file for 2.13.3

With the security issues text and a list of minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMerge branch 'stable-2.12' into stable-2.13
Hrvoje Ribicic [Mon, 14 Dec 2015 17:33:14 +0000 (18:33 +0100)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Bump revision number for 2.12.6
  Update NEWS file for 2.12.6

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Merge entries
  configure.ac - Take 2.13 version numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoBump revision number for 2.12.6 v2.12.6
Hrvoje Ribicic [Mon, 14 Dec 2015 16:42:03 +0000 (17:42 +0100)]
Bump revision number for 2.12.6

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoUpdate NEWS file for 2.12.6
Hrvoje Ribicic [Mon, 14 Dec 2015 16:41:09 +0000 (17:41 +0100)]
Update NEWS file for 2.12.6

With the security issues text and a list of minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.11' into stable-2.12
Hrvoje Ribicic [Mon, 14 Dec 2015 16:15:14 +0000 (17:15 +0100)]
Merge branch 'stable-2.11' into stable-2.12

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Merged entries
  configure.ac - Took 2.12 version numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoRevision bump for 2.11.8 v2.11.8
Hrvoje Ribicic [Mon, 14 Dec 2015 14:07:23 +0000 (15:07 +0100)]
Revision bump for 2.11.8

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoUpdate NEWS file for 2.11.8
Hrvoje Ribicic [Mon, 14 Dec 2015 14:06:50 +0000 (15:06 +0100)]
Update NEWS file for 2.11.8

With the security issues text and a list of minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoFix error message in attachInstanceDiskChecks
Lisa Velden [Mon, 14 Dec 2015 14:13:10 +0000 (15:13 +0100)]
Fix error message in attachInstanceDiskChecks

Name the instance where disks are already attached to, which is not
necessarily the instance where we want to attach a disk to.
This fixes issue 1151.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMerge branch 'stable-2.10' into stable-2.11
Hrvoje Ribicic [Mon, 14 Dec 2015 13:13:03 +0000 (14:13 +0100)]
Merge branch 'stable-2.10' into stable-2.11

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Combine NEWS entries from both versions
  configure.ac - Take correct version numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoVersion bump for 2.10.8 v2.10.8
Hrvoje Ribicic [Fri, 11 Dec 2015 11:09:21 +0000 (12:09 +0100)]
Version bump for 2.10.8

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoUpdate NEWS file for 2.10.8
Hrvoje Ribicic [Fri, 11 Dec 2015 11:08:22 +0000 (12:08 +0100)]
Update NEWS file for 2.10.8

With the security issues text and list minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.9' into stable-2.10
Hrvoje Ribicic [Thu, 10 Dec 2015 18:04:48 +0000 (19:04 +0100)]
Merge branch 'stable-2.9' into stable-2.10

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - leave 2.9.7 info in
  configure.ac - revert version bump

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoBump revision number stable-2.9 v2.9.7
Hrvoje Ribicic [Thu, 10 Dec 2015 16:40:51 +0000 (17:40 +0100)]
Bump revision number

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUpdate NEWS file for 2.9.7 release
Hrvoje Ribicic [Thu, 10 Dec 2015 16:39:53 +0000 (17:39 +0100)]
Update NEWS file for 2.9.7 release

... with security release info and minor changes.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoImprove RAPI section on security
Hrvoje Ribicic [Thu, 10 Dec 2015 13:22:01 +0000 (14:22 +0100)]
Improve RAPI section on security

The RAPI section on security has been improved with new information
related on how users can lock RAPI down as they see fit, and what are
the risks involved with default settings.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoUpdate documentation of harep
Klaus Aehlig [Thu, 3 Dec 2015 10:24:31 +0000 (11:24 +0100)]
Update documentation of harep

Be more explicit about which action is taken by harep under
which conditions. In particular, mention the limitation that
harep never carries out migration operations.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoDocument harep --dry-run in the man page
Klaus Aehlig [Thu, 3 Dec 2015 08:52:32 +0000 (09:52 +0100)]
Document harep --dry-run in the man page

Document the new --dry-run option in harep's man page.
Also mention the limitations, as harep records its state
in instance tags.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoSupport --dry-run in harep
Klaus Aehlig [Wed, 2 Dec 2015 16:20:46 +0000 (17:20 +0100)]
Support --dry-run in harep

Add a --dry-run option to harep, so that users can verify
that the actions taken by harep are the ones they want.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoAdd a --dry-run option to htools
Klaus Aehlig [Wed, 2 Dec 2015 13:51:56 +0000 (14:51 +0100)]
Add a --dry-run option to htools

Add a new flag, --dry-run, to the available flags in htools.
It will be used for harep to allow diagnose-only runs.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.14' into stable-2.15
Hrvoje Ribicic [Fri, 4 Dec 2015 15:06:50 +0000 (16:06 +0100)]
Merge branch 'stable-2.14' into stable-2.15

* stable-2.14
  Fix lines with more than 80 characters
  Add more detach/attach sequence tests
  Allow disk attachment to diskless instances
  Improve tests for attaching disks

* stable-2.13
  (no changes)

* stable-2.12
  Restrict showing of DRBD secret using types
  Calculate correct affected nodes set in InstanceChangeGroup

* stable-2.11
  (no changes)

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.13' into stable-2.14
Hrvoje Ribicic [Thu, 3 Dec 2015 22:55:20 +0000 (22:55 +0000)]
Merge branch 'stable-2.13' into stable-2.14

* stable-2.13
  (no changes)

* stable-2.12
  Restrict showing of DRBD secret using types
  Calculate correct affected nodes set in InstanceChangeGroup

* stable-2.11
  (no changes)

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  src/Ganeti/Objects.hs - Followed code to Disk.hs
  test/hs/Test/Ganeti/Objects.hs - Added Private to disk definition

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoMerge branch 'stable-2.12' into stable-2.13
Hrvoje Ribicic [Thu, 3 Dec 2015 21:13:39 +0000 (21:13 +0000)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Restrict showing of DRBD secret using types
  Calculate correct affected nodes set in InstanceChangeGroup

* stable-2.11
  (no changes)

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoRestrict showing of DRBD secret using types
Hrvoje Ribicic [Tue, 1 Dec 2015 16:11:38 +0000 (16:11 +0000)]
Restrict showing of DRBD secret using types

While the Python changes from 2.9 do prevent Ganeti from accidentally
revealing the Haskell secret, they may not do so forever. The queries
are planned to switch from Python to Haskell at some point, and should
someone want to use the DRBD secret, they can do so easily.

As a more elegant way of hiding the secret, wrap it in a Private
wrapper, preventing it from leaking out unless explicitly requested.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.11' into stable-2.12
Hrvoje Ribicic [Tue, 1 Dec 2015 15:57:49 +0000 (15:57 +0000)]
Merge branch 'stable-2.11' into stable-2.12

* stable-2.11
  (no changes)

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  lib/client/gnt_instance.py - taken the 2.11 version, with explicit
                               parameter use
  qa/qa_rapi.py - merged imports, resolved trivial conflict

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.10' into stable-2.11
Hrvoje Ribicic [Mon, 30 Nov 2015 16:12:42 +0000 (17:12 +0100)]
Merge branch 'stable-2.10' into stable-2.11

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  qa/qa_rapi.py - simply append new changes

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.9' into stable-2.10
Hrvoje Ribicic [Mon, 30 Nov 2015 15:49:09 +0000 (16:49 +0100)]
Merge branch 'stable-2.9' into stable-2.10

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  lib/cmdlib/instance_query.py - removed physical_id changes

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoQA: Ensure the DRBD secret is not retrievable via RAPI
Hrvoje Ribicic [Fri, 27 Nov 2015 17:32:42 +0000 (17:32 +0000)]
QA: Ensure the DRBD secret is not retrievable via RAPI

The best way to ensure that the DRBD secret does not inadvertently leak
is to introduce a QA test examining the output of the interface in
which the leak was originally introduced.

The test added determines the DRBD secret and makes RAPI requests,
examining them for its presence and failing if a match is found.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoRedact the DRBD secret in instance queries
Hrvoje Ribicic [Fri, 27 Nov 2015 15:58:13 +0000 (15:58 +0000)]
Redact the DRBD secret in instance queries

As the DRBD secret should be used only by Ganeti internals, replacing
the actual secret with None does not hamper Ganeti's work, while
preventing the secret from being leaked.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoDo not attempt to use the DRBD secret in gnt-instance info
Hrvoje Ribicic [Fri, 21 Aug 2015 19:46:18 +0000 (19:46 +0000)]
Do not attempt to use the DRBD secret in gnt-instance info

... so just redact what is output.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoFix lines with more than 80 characters
Lisa Velden [Fri, 27 Nov 2015 10:25:55 +0000 (11:25 +0100)]
Fix lines with more than 80 characters

Previous refactoring has introduced lines with too many characters.
This patch fixes this.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoFix lines with more than 80 characters
Lisa Velden [Fri, 27 Nov 2015 10:25:55 +0000 (11:25 +0100)]
Fix lines with more than 80 characters

Previous refactoring has introduced lines with too many characters.
This patch fixes this.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoAdd more detach/attach sequence tests
Lisa Velden [Wed, 25 Nov 2015 16:57:18 +0000 (17:57 +0100)]
Add more detach/attach sequence tests

Test detach/attach sequences with an instance that becomes diskless
after detaching its disk and also test detach/attach with drbd disks.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoAllow disk attachment to diskless instances
Lisa Velden [Wed, 25 Nov 2015 15:00:45 +0000 (16:00 +0100)]
Allow disk attachment to diskless instances

As only DRBD disks can be associated to more nodes than the instance
where we want to attach the disk to, we have to change the check for
associated nodes, too.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>