ganeti-github.git
4 years agossh_update: log data that is received
Helga Velroyen [Tue, 15 Dec 2015 14:03:53 +0000 (15:03 +0100)]
ssh_update: log data that is received

Debugging ssh_update can be annoying, because the data
used as input is not dumped anywhere. This patch logs
makes sure it gets logged (at DEBUG level) when
ssh_update receives the data.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Cherry-picked-from: 5c370ec180
Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoIncrease timeout of RPC adding/removing keys
Helga Velroyen [Thu, 7 Jan 2016 13:27:29 +0000 (14:27 +0100)]
Increase timeout of RPC adding/removing keys

This patch increases the timeout for the RPC calls that
add and remove SSH keys to the cluster. This is necessary,
because in big clusters the distribution/removal of a
key takes too long as Ganeti has to contact every node in
the cluster.

This patch increases the timeout from URGENT to FAST
(the next higher option).

The alternatives to this include splitting up the
RPC call to several calls, which will add addiional
overall runtime and RPC overhead as well as security
implications. Since the higher timeout was tested
in a big cluster, we go with this for now.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoAfter TestNodeModify, fix the pool of master candidates
Klaus Aehlig [Tue, 22 Dec 2015 11:35:40 +0000 (12:35 +0100)]
After TestNodeModify, fix the pool of master candidates

The test TestNodeModify temporarily modifies the cluster parameter
candidate-pool-size, which controls the minimal desirable number of
master candidates. Depending on the size of the test cluster, this
temporary modification can be a decrease (for clusters with up to 10
nodes) or an increase (for clusters with 12 or more nodes). Ganeti's
behavior upon change of the candidate pool size is to promote nodes to
master candidates upon increase, but do nothing upon decrease. This is
a safe behavior, as too many master candidates is not a problem; the
chance of data loss is even smaller. However, it means that the test
has a size effect of, for large test cluster, increasing the actual
number of nodes that are master candidates. While not a problem for
correctness, this side effect does affect our performance tests (which
usually are run after the functional tests) as more master candidates
means more nodes to replicate information to.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoAdd more documentation to testutils_ssh.py
Helga Velroyen [Tue, 1 Dec 2015 15:20:57 +0000 (16:20 +0100)]
Add more documentation to testutils_ssh.py

This patch adds more comments to the functions in
testutils_ssh.py, in particular to clarify which function
returns what types of objects.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agorenew-crypto: use bulk-removal of SSH keys
Helga Velroyen [Tue, 24 Nov 2015 12:01:46 +0000 (13:01 +0100)]
renew-crypto: use bulk-removal of SSH keys

This patch makes renew-crypto use the newly introduced
bulk-removal function for SSH keys. This way the
complexity of renew-crypto (in terms of number of
SSH connections) becomes linear (from previously
quadratic).

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoUse bulk-removal of SSH keys for single keys
Helga Velroyen [Tue, 24 Nov 2015 10:33:29 +0000 (11:33 +0100)]
Use bulk-removal of SSH keys for single keys

As the code for bulk-removal of SSH keys subsumes
the code for removing a single SSH key, let the
latter call the first.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoBulk-removing SSH keys of diverse set of nodes
Helga Velroyen [Fri, 20 Nov 2015 10:16:58 +0000 (11:16 +0100)]
Bulk-removing SSH keys of diverse set of nodes

This patch adds a unit test where SSH keys of a diverse
set of nodes is removed. By 'diverse', we mean a set
consisting of master candidates, potential master
candidates, and normal nodes.

It also fixes some minor bug that surfaced with that
test.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoBulk-removal of SSH keys of normal nodes
Helga Velroyen [Fri, 20 Nov 2015 09:41:12 +0000 (10:41 +0100)]
Bulk-removal of SSH keys of normal nodes

This patch adds a unit test for bulk-removing
normal nodes. Besides that, it fixes a small
bug that surfaced with that test.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoBulk-remove SSH keys of potential master candidates
Helga Velroyen [Fri, 20 Nov 2015 09:30:08 +0000 (10:30 +0100)]
Bulk-remove SSH keys of potential master candidates

This patch adds a unit test for bulk-removing potential
master candidates.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoBulk-removal of SSH keys
Helga Velroyen [Fri, 20 Nov 2015 09:11:44 +0000 (10:11 +0100)]
Bulk-removal of SSH keys

In order to improve the runtime complexity of
'renew-crypto', this patch adds a function to
bulk-remove SSH keys of nodes (in contrast to
the function that only removes one key at a time).

Within this patch, it is only called in a unit
test. Further patches will integrate and test it
further.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agotestutils: add keys to own 'authorized_keys' file
Helga Velroyen [Tue, 24 Nov 2015 10:11:41 +0000 (11:11 +0100)]
testutils: add keys to own 'authorized_keys' file

This patch updates the SSH testutils to match reality better.
So far, the test framework did not consider the fact that
the key of each node should be added to it's own
'authorized_keys' file, even if the node is not a master
candidate. This patch fixes that to represent the production
behavior more accurately.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMake mock SSH file manager deal with lists
Helga Velroyen [Thu, 19 Nov 2015 15:13:17 +0000 (16:13 +0100)]
Make mock SSH file manager deal with lists

There was a subtle bug in the unit test of backend.py
which was masking another subtle bug in the test framework
in testutils_ssh.py.

As relict from some previous refactoring, the ssh.py
functions assume that there can be more than one public
key per node. The testutils so far assume there is only
one key per node and due to a bug, this cancelled out
nicely and was not found so far.

As we actually only have one key per node, the elegant
thing to do would be to adapt ssh.py rather than the
testutils, but that will break the interface of the
ssh_update.py tool. Since we would rather not do that
in a stable, branch, this patch adapts the testutils.
The adaption of the ssh.py will be done in a newer
branch then.

Additionally, this patch also sprinkles assertions
everywhere to ensure finding these kind of type messups
sooner.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoDon't deepcopy the config if the old value is not needed
Klaus Aehlig [Mon, 14 Dec 2015 14:08:22 +0000 (15:08 +0100)]
Don't deepcopy the config if the old value is not needed

The _UpgradeConfig function carries out internal upgrades of the
configuration, and additionally, if requested, saves the configuration
in case it changed in this process. To compare the old and the new
version, a deep copy of the old version is kept. As deep copying large
configurations is an expensive operation, only do it, if the value is
used afterwards.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoRevision bump for 2.15.2 v2.15.2
Hrvoje Ribicic [Wed, 16 Dec 2015 12:16:57 +0000 (12:16 +0000)]
Revision bump for 2.15.2

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUpdate NEWS file for 2.15.2
Hrvoje Ribicic [Wed, 16 Dec 2015 12:16:39 +0000 (12:16 +0000)]
Update NEWS file for 2.15.2

With the security information and a list of minor changes.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMerge branch 'stable-2.14' into stable-2.15
Hrvoje Ribicic [Wed, 16 Dec 2015 11:09:38 +0000 (12:09 +0100)]
Merge branch 'stable-2.14' into stable-2.15

* stable-2.14
  Revision bump for 2.14.2
  Update NEWS file for 2.14.2

* stable-2.13
  Revision bump for 2.13.3
  Update NEWS file for 2.13.3

* stable-2.12
  Bump revision number for 2.12.6
  Update NEWS file for 2.12.6

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Merge entries
  configure.ac - Take 2.15 revision numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoRevision bump for 2.14.2 v2.14.2
Hrvoje Ribicic [Tue, 15 Dec 2015 17:54:17 +0000 (18:54 +0100)]
Revision bump for 2.14.2

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUpdate NEWS file for 2.14.2
Hrvoje Ribicic [Tue, 15 Dec 2015 17:53:11 +0000 (18:53 +0100)]
Update NEWS file for 2.14.2

With the security issues text and a list of minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMerge branch 'stable-2.13' into stable-2.14
Hrvoje Ribicic [Tue, 15 Dec 2015 14:44:16 +0000 (15:44 +0100)]
Merge branch 'stable-2.13' into stable-2.14

* stable-2.13
  Revision bump for 2.13.3
  Update NEWS file for 2.13.3

* stable-2.12
  Bump revision number for 2.12.6
  Update NEWS file for 2.12.6

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Merged entries
  configure.ac - Took 2.14 version numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoRevision bump for 2.13.3 v2.13.3
Hrvoje Ribicic [Mon, 14 Dec 2015 18:00:43 +0000 (19:00 +0100)]
Revision bump for 2.13.3

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUpdate NEWS file for 2.13.3
Hrvoje Ribicic [Mon, 14 Dec 2015 17:59:26 +0000 (18:59 +0100)]
Update NEWS file for 2.13.3

With the security issues text and a list of minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMerge branch 'stable-2.12' into stable-2.13
Hrvoje Ribicic [Mon, 14 Dec 2015 17:33:14 +0000 (18:33 +0100)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Bump revision number for 2.12.6
  Update NEWS file for 2.12.6

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Merge entries
  configure.ac - Take 2.13 version numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoBump revision number for 2.12.6 v2.12.6
Hrvoje Ribicic [Mon, 14 Dec 2015 16:42:03 +0000 (17:42 +0100)]
Bump revision number for 2.12.6

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoUpdate NEWS file for 2.12.6
Hrvoje Ribicic [Mon, 14 Dec 2015 16:41:09 +0000 (17:41 +0100)]
Update NEWS file for 2.12.6

With the security issues text and a list of minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.11' into stable-2.12
Hrvoje Ribicic [Mon, 14 Dec 2015 16:15:14 +0000 (17:15 +0100)]
Merge branch 'stable-2.11' into stable-2.12

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Merged entries
  configure.ac - Took 2.12 version numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoRevision bump for 2.11.8 v2.11.8
Hrvoje Ribicic [Mon, 14 Dec 2015 14:07:23 +0000 (15:07 +0100)]
Revision bump for 2.11.8

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoUpdate NEWS file for 2.11.8
Hrvoje Ribicic [Mon, 14 Dec 2015 14:06:50 +0000 (15:06 +0100)]
Update NEWS file for 2.11.8

With the security issues text and a list of minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.10' into stable-2.11
Hrvoje Ribicic [Mon, 14 Dec 2015 13:13:03 +0000 (14:13 +0100)]
Merge branch 'stable-2.10' into stable-2.11

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Combine NEWS entries from both versions
  configure.ac - Take correct version numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoVersion bump for 2.10.8 v2.10.8
Hrvoje Ribicic [Fri, 11 Dec 2015 11:09:21 +0000 (12:09 +0100)]
Version bump for 2.10.8

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoUpdate NEWS file for 2.10.8
Hrvoje Ribicic [Fri, 11 Dec 2015 11:08:22 +0000 (12:08 +0100)]
Update NEWS file for 2.10.8

With the security issues text and list minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.9' into stable-2.10
Hrvoje Ribicic [Thu, 10 Dec 2015 18:04:48 +0000 (19:04 +0100)]
Merge branch 'stable-2.9' into stable-2.10

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - leave 2.9.7 info in
  configure.ac - revert version bump

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoBump revision number stable-2.9 v2.9.7
Hrvoje Ribicic [Thu, 10 Dec 2015 16:40:51 +0000 (17:40 +0100)]
Bump revision number

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUpdate NEWS file for 2.9.7 release
Hrvoje Ribicic [Thu, 10 Dec 2015 16:39:53 +0000 (17:39 +0100)]
Update NEWS file for 2.9.7 release

... with security release info and minor changes.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoImprove RAPI section on security
Hrvoje Ribicic [Thu, 10 Dec 2015 13:22:01 +0000 (14:22 +0100)]
Improve RAPI section on security

The RAPI section on security has been improved with new information
related on how users can lock RAPI down as they see fit, and what are
the risks involved with default settings.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.14' into stable-2.15
Hrvoje Ribicic [Fri, 4 Dec 2015 15:06:50 +0000 (16:06 +0100)]
Merge branch 'stable-2.14' into stable-2.15

* stable-2.14
  Fix lines with more than 80 characters
  Add more detach/attach sequence tests
  Allow disk attachment to diskless instances
  Improve tests for attaching disks

* stable-2.13
  (no changes)

* stable-2.12
  Restrict showing of DRBD secret using types
  Calculate correct affected nodes set in InstanceChangeGroup

* stable-2.11
  (no changes)

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.13' into stable-2.14
Hrvoje Ribicic [Thu, 3 Dec 2015 22:55:20 +0000 (22:55 +0000)]
Merge branch 'stable-2.13' into stable-2.14

* stable-2.13
  (no changes)

* stable-2.12
  Restrict showing of DRBD secret using types
  Calculate correct affected nodes set in InstanceChangeGroup

* stable-2.11
  (no changes)

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  src/Ganeti/Objects.hs - Followed code to Disk.hs
  test/hs/Test/Ganeti/Objects.hs - Added Private to disk definition

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoMerge branch 'stable-2.12' into stable-2.13
Hrvoje Ribicic [Thu, 3 Dec 2015 21:13:39 +0000 (21:13 +0000)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Restrict showing of DRBD secret using types
  Calculate correct affected nodes set in InstanceChangeGroup

* stable-2.11
  (no changes)

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoRestrict showing of DRBD secret using types
Hrvoje Ribicic [Tue, 1 Dec 2015 16:11:38 +0000 (16:11 +0000)]
Restrict showing of DRBD secret using types

While the Python changes from 2.9 do prevent Ganeti from accidentally
revealing the Haskell secret, they may not do so forever. The queries
are planned to switch from Python to Haskell at some point, and should
someone want to use the DRBD secret, they can do so easily.

As a more elegant way of hiding the secret, wrap it in a Private
wrapper, preventing it from leaking out unless explicitly requested.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.11' into stable-2.12
Hrvoje Ribicic [Tue, 1 Dec 2015 15:57:49 +0000 (15:57 +0000)]
Merge branch 'stable-2.11' into stable-2.12

* stable-2.11
  (no changes)

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  lib/client/gnt_instance.py - taken the 2.11 version, with explicit
                               parameter use
  qa/qa_rapi.py - merged imports, resolved trivial conflict

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.10' into stable-2.11
Hrvoje Ribicic [Mon, 30 Nov 2015 16:12:42 +0000 (17:12 +0100)]
Merge branch 'stable-2.10' into stable-2.11

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  qa/qa_rapi.py - simply append new changes

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.9' into stable-2.10
Hrvoje Ribicic [Mon, 30 Nov 2015 15:49:09 +0000 (16:49 +0100)]
Merge branch 'stable-2.9' into stable-2.10

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  lib/cmdlib/instance_query.py - removed physical_id changes

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoQA: Ensure the DRBD secret is not retrievable via RAPI
Hrvoje Ribicic [Fri, 27 Nov 2015 17:32:42 +0000 (17:32 +0000)]
QA: Ensure the DRBD secret is not retrievable via RAPI

The best way to ensure that the DRBD secret does not inadvertently leak
is to introduce a QA test examining the output of the interface in
which the leak was originally introduced.

The test added determines the DRBD secret and makes RAPI requests,
examining them for its presence and failing if a match is found.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoRedact the DRBD secret in instance queries
Hrvoje Ribicic [Fri, 27 Nov 2015 15:58:13 +0000 (15:58 +0000)]
Redact the DRBD secret in instance queries

As the DRBD secret should be used only by Ganeti internals, replacing
the actual secret with None does not hamper Ganeti's work, while
preventing the secret from being leaked.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoDo not attempt to use the DRBD secret in gnt-instance info
Hrvoje Ribicic [Fri, 21 Aug 2015 19:46:18 +0000 (19:46 +0000)]
Do not attempt to use the DRBD secret in gnt-instance info

... so just redact what is output.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

5 years agoFix lines with more than 80 characters
Lisa Velden [Fri, 27 Nov 2015 10:25:55 +0000 (11:25 +0100)]
Fix lines with more than 80 characters

Previous refactoring has introduced lines with too many characters.
This patch fixes this.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

5 years agoAdd more detach/attach sequence tests
Lisa Velden [Wed, 25 Nov 2015 16:57:18 +0000 (17:57 +0100)]
Add more detach/attach sequence tests

Test detach/attach sequences with an instance that becomes diskless
after detaching its disk and also test detach/attach with drbd disks.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

5 years agoAllow disk attachment to diskless instances
Lisa Velden [Wed, 25 Nov 2015 15:00:45 +0000 (16:00 +0100)]
Allow disk attachment to diskless instances

As only DRBD disks can be associated to more nodes than the instance
where we want to attach the disk to, we have to change the check for
associated nodes, too.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

5 years agoImprove tests for attaching disks
Lisa Velden [Wed, 25 Nov 2015 13:53:39 +0000 (14:53 +0100)]
Improve tests for attaching disks

by associating disks and instances to a specific node.
Also refactor mock uuids and mock disk names into variables.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

5 years agoCompute lock allocation strictly
Klaus Aehlig [Thu, 26 Nov 2015 16:49:38 +0000 (17:49 +0100)]
Compute lock allocation strictly

Given that on updates it has to be fully computed anyway, do not
accumulate thunks during the computation.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

5 years agoCalculate correct affected nodes set in InstanceChangeGroup
Oleg Ponomarev [Fri, 20 Nov 2015 20:45:11 +0000 (21:45 +0100)]
Calculate correct affected nodes set in InstanceChangeGroup

This is the fix for the issue 1144. The nodes affected by the
InstanceChangeGroup logical unit were calculated incorrectly and that
broke 'gnt-instance change-group --to' operation. This patch fixes it.

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

5 years agoDocument the decission why optimisation is turned off
Klaus Aehlig [Thu, 19 Nov 2015 13:27:03 +0000 (14:27 +0100)]
Document the decission why optimisation is turned off

Commit c22a35 removed an argument of readJSONWithDesc which
caused some versions of ghc to go too crazy in optimising,
so it had to be turned off for some files. Document that reason
in a comment.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

5 years agoDon't keep input for error messages
Klaus Aehlig [Wed, 18 Nov 2015 13:59:36 +0000 (14:59 +0100)]
Don't keep input for error messages

When generating error messages, the raw JSValue is rarely
useful. However, keeping it for error messages---even if
only in the unused branch of an if statement---prevents this
value from going out of scope.

Note: with the smaller number of arguments in the readJSONWithDesc
function, newer versions of ghc try too fancy optimisations and thus
run out of memory; hence we have to reduce the ghc optimisation level
for some files.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

5 years agoUse dict.copy instead of deepcopy
Helga Velroyen [Wed, 18 Nov 2015 08:44:43 +0000 (09:44 +0100)]
Use dict.copy instead of deepcopy

Due to a bug in python, deepcopy does not work on
the dictionaries we use for SSH updates. This patch
replaces the use of deepcopy by the built-in copy
function of dictionaries.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

5 years agoUse bulk-adding of keys in renew-crypto
Helga Velroyen [Thu, 12 Nov 2015 12:48:59 +0000 (13:48 +0100)]
Use bulk-adding of keys in renew-crypto

This patch makes renew-crypto actually use the bulk-adding
function of SSH keys rather than adding each key
individually. This patch also adds a unit tests where the
bulk-adding is tested with a diverse set of keys (master
candidates, potential master candidates, normal nodes).

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

5 years agoMake NodeSshKeyAdd use its *Bulk companion
Helga Velroyen [Thu, 12 Nov 2015 10:21:45 +0000 (11:21 +0100)]
Make NodeSshKeyAdd use its *Bulk companion

Since the bulk-version of adding keys is subsuming the
functionality of adding a single key, this patch makes
NodeSshKeyAdd internally use the *Bulk version. The
unit tests in place make sure no functionality is
changed.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

5 years agoUnit test bulk-adding normal nodes
Helga Velroyen [Thu, 12 Nov 2015 10:06:26 +0000 (11:06 +0100)]
Unit test bulk-adding normal nodes

This patch adds a unit test that tests adding a bulk
of normal nodes' SSH keys to the cluster.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

5 years agoUnit test for bulk-adding pot. master candidates
Helga Velroyen [Thu, 12 Nov 2015 10:05:15 +0000 (11:05 +0100)]
Unit test for bulk-adding pot. master candidates

This patch adds a unit tests for bulk-adding SSH keys
of potential master candidates.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

5 years agoIntroduce bulk-adding of SSH keys
Helga Velroyen [Wed, 11 Nov 2015 15:54:31 +0000 (16:54 +0100)]
Introduce bulk-adding of SSH keys

This patch introduces a backend function to add a set of
SSH keys to the nodes (rather than one key at a time).
The bulk-adding function is having the same structure
as the original one, but is adapted to work with a set
of keys rather than one key.

This patch also adds a unit test for testing the
bulk-adding of keys.

Note that this patch only adds the bulk-adding function
but does not use it yet. In the following patches of
this series, we will add more unit tests and at the
end integrate the bulk-adding function into
renew-crypto.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

5 years agoPause watcher during performance QA
Klaus Aehlig [Tue, 17 Nov 2015 14:16:28 +0000 (15:16 +0100)]
Pause watcher during performance QA

Our performance QA tests are intended to alert us if some common
task suddenly takes longer. To serve this purpose, they need to provide
reproducible results. Hence avoid any interference with watcher-submitted
jobs by pausing the watcher during performance QA tests.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

5 years agoSend answers strictly
Klaus Aehlig [Mon, 16 Nov 2015 14:05:45 +0000 (15:05 +0100)]
Send answers strictly

When sending an answer over a domain socket, the recipient
won't process that answer anyway before it is complete. So
we can as well assemble one ByteString first and send it over
the wire all at once, thus saving a few system calls.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

5 years agoStore keys as ByteStrings
Klaus Aehlig [Thu, 12 Nov 2015 13:51:16 +0000 (14:51 +0100)]
Store keys as ByteStrings

Keys to maps are only used to look up values, so
a compact representation does impact flexibility.
However, it does save on memory usage; having more
locality in the keys also improves time when comparing
them.

While there, also refrain from linearly looking through
keys searching for partial matches where partial matches
are not desired (e.g., when looking up things by uuid).

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

5 years agoMerge branch 'stable-2.14' into stable-2.15
Klaus Aehlig [Thu, 12 Nov 2015 10:45:16 +0000 (11:45 +0100)]
Merge branch 'stable-2.14' into stable-2.15

* stable-2.14
  Fix faulty iallocator type check
  Improve cfgupgrade output in case of errors

* stable-2.13
  Extend timeout for gnt-cluster renew-crypto
  Reduce flakyness of GetCmdline test on slow machines
  Remove duplicated words

* stable-2.12
  Revert "Also consider connection time out a network error"
  Clone lists before modifying
  Make lockConfig call retryable
  Return the correct error code in the post-upgrade script
  Make openssl refrain from DH altogether
  Fix upgrades of instances with missing creation time

* stable-2.11
  (no changes)

* stable-2.10
  Remove -X from hspace man page
  Make htools tolerate missing "dtotal" and "dfree" on luxi

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

5 years agoEncode UUIDs as ByteStrings
Klaus Aehlig [Wed, 11 Nov 2015 11:07:03 +0000 (12:07 +0100)]
Encode UUIDs as ByteStrings

UUIDs are fixed-length strings at which we either look
completely or not at all. Moreover, we do not do any
computations on them. Therefore, we can chose a more
compact representation on them, resulting in reduced memory
foot print.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

5 years agoPrefer the UuidObject type class over specific functions
Klaus Aehlig [Wed, 11 Nov 2015 11:26:24 +0000 (12:26 +0100)]
Prefer the UuidObject type class over specific functions

The UuidObject type class provides a clean interface to
obtain the UUID of an object. Prefer this interface over
hard-coding the specific functions all over the place.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

5 years agoMerge branch 'stable-2.13' into stable-2.14
Oleg Ponomarev [Wed, 11 Nov 2015 18:01:36 +0000 (19:01 +0100)]
Merge branch 'stable-2.13' into stable-2.14

* stable-2.13
  Extend timeout for gnt-cluster renew-crypto

* stable-2.12
  Revert "Also consider connection time out a network error"
  Clone lists before modifying
  Make lockConfig call retryable

* stable-2.11
  (no changes)

* stable-2.10
  Remove -X from hspace man page
  Make htools tolerate missing "dtotal" and "dfree" on luxi

Conflicts:
    tools/cfgupgrade
Resolution
    take the change into lib/tools/cfgupgrade

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

5 years agoMerge branch 'stable-2.12' into stable-2.13
Oleg Ponomarev [Wed, 11 Nov 2015 17:14:51 +0000 (18:14 +0100)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Revert "Also consider connection time out a network error"
  Clone lists before modifying
  Make lockConfig call retryable

* stable-2.11
  (no changes)

* stable-2.10
  Remove -X from hspace man page
  Make htools tolerate missing "dtotal" and "dfree" on luxi

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

5 years agoMerge branch 'stable-2.11' into stable-2.12
Oleg Ponomarev [Wed, 11 Nov 2015 16:04:40 +0000 (17:04 +0100)]
Merge branch 'stable-2.11' into stable-2.12

    * stable-2.11
      (no changes)

    * stable-2.10
      Remove -X from hspace man page
      Make htools tolerate missing "dtotal" and "dfree" on luxi

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Reviewed-by: Liza Velden <velden@google.com>

5 years agoMerge branch 'stable-2.10' into stable-2.11
Klaus Aehlig [Wed, 11 Nov 2015 15:51:42 +0000 (16:51 +0100)]
Merge branch 'stable-2.10' into stable-2.11

* stable-2.10
  Remove -X from hspace man page
  Make htools tolerate missing "dtotal" and "dfree" on luxi

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

5 years agoRevert "Also consider connection time out a network error"
Klaus Aehlig [Tue, 10 Nov 2015 16:47:44 +0000 (17:47 +0100)]
Revert "Also consider connection time out a network error"

This reverts commit 84c17185ad47070944c64ab64a8c7dfd60a260f9.
We use RetryOnNetworkError for basically every form of internal
communication. While it makes sense to retry---given that we
assume daemons might come and go at any time---we can only do
so safely, if we positively know that we did not cause any
side effect. Given that not all our requests are idempotent
(e.g., submitting jobs is not)---in fact, the majority is
not--, retrying on timeouts is not safe.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

5 years agoClone lists before modifying
Klaus Aehlig [Tue, 10 Nov 2015 15:40:47 +0000 (16:40 +0100)]
Clone lists before modifying

When an opcode expands to a list of jobs, we extend the reason trail
of the new jobs with that of the original opcode that expanded to them.
Before modifying the reason trail, however, we should duplicate it to
avoid side effects on shared copies---like the default empty list.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

5 years agoAssign the variables before use (bugfix for dee6adb9)
Oleg Ponomarev [Mon, 9 Nov 2015 16:28:38 +0000 (17:28 +0100)]
Assign the variables before use (bugfix for dee6adb9)

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

5 years agoExtend QA to detect autopromotion errors
Helga Velroyen [Fri, 6 Nov 2015 10:27:41 +0000 (11:27 +0100)]
Extend QA to detect autopromotion errors

The issue that was fixed with the previous patch would
have been detected earlier if the QA would actually
run a 'verify' after the modify operations. For 'verify'
to not raise false negatives, we need to first reduce
the candidate pool size, because otherwise QA fails
with a warning about the mininmum pool size being
violated.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

5 years agoHandle SSH key distribution on auto promotion
Helga Velroyen [Fri, 6 Nov 2015 10:26:08 +0000 (11:26 +0100)]
Handle SSH key distribution on auto promotion

This fixes the missing SSH key distribution in case
a node gets autopromoted to master candidate.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

5 years agoDo not remove authorized key of node itself
Helga Velroyen [Fri, 6 Nov 2015 09:11:19 +0000 (10:11 +0100)]
Do not remove authorized key of node itself

This fixes a small bug that if a node was demoted
from master candidate, that its own public key
was removed from its own authorized key file.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

5 years agoFix indentation
Lisa Velden [Thu, 5 Nov 2015 10:36:57 +0000 (11:36 +0100)]
Fix indentation

so that the method can be called correctly.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

5 years agoMake lockConfig call retryable
Klaus Aehlig [Wed, 4 Nov 2015 13:52:16 +0000 (14:52 +0100)]
Make lockConfig call retryable

Locking the configuration is naturally idempotent. However,
the corresponding WConfD call had a check refusing to lock
the config, if the caller has already locked it, arguing that
this should not happen. That argument misses that we have the
built-in assumption that daemons might be restarted at any time,
including the moment where a request is processed, but the caller
did not get the answer yet. So allow retries, hower logging that
they occurred (as this should only happen rarely).

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

5 years agoExtend timeout for gnt-cluster renew-crypto
Hrvoje Ribicic [Wed, 4 Nov 2015 13:01:38 +0000 (14:01 +0100)]
Extend timeout for gnt-cluster renew-crypto

With particularly large clusters, the renewal of SSH keys happening in
renew-crypto can take a long time to complete. While this should be
improved, an additional problem is that the RPC doing most of the work
has a default one-hour timeout. Given that it is preferable that the
operation completes, this patch bumps the timeout to four hours, which
should suffice even for 80+ node clusters.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

5 years agoMerge branch 'stable-2.12' into stable-2.13
Hrvoje Ribicic [Mon, 2 Nov 2015 17:49:36 +0000 (17:49 +0000)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Return the correct error code in the post-upgrade script
  Make openssl refrain from DH altogether
  Fix upgrades of instances with missing creation time

Conflicts:
cfgupgrade_unittest.py: merge version tests
tools/post-upgrade: return the correct error code for SSH
                    renewal as well

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

5 years agoReturn the correct error code in the post-upgrade script
Hrvoje Ribicic [Mon, 2 Nov 2015 17:19:22 +0000 (17:19 +0000)]
Return the correct error code in the post-upgrade script

While we want all the post-upgrade actions to be undertaken, should one
of these fail, the correct error code should be returned so that the
upgrade script can report issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

5 years agoMake openssl refrain from DH altogether
Klaus Aehlig [Mon, 2 Nov 2015 10:44:34 +0000 (11:44 +0100)]
Make openssl refrain from DH altogether

As various ssl implementations have different ideas about
which dh key lengths are acceptable, refrain from standard
dh altogether (and not only from anonymous dh) to avoid
handshake problems.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

5 years agoRemove -X from hspace man page
Klaus Aehlig [Mon, 26 Oct 2015 12:34:17 +0000 (13:34 +0100)]
Remove -X from hspace man page

hspace never had such an option.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

Cherry-picked-from: fa36daf4
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

5 years agoFix faulty iallocator type check
Hrvoje Ribicic [Wed, 28 Oct 2015 17:56:23 +0000 (17:56 +0000)]
Fix faulty iallocator type check

Because the ignore-soft-errors parameter is optional rather than always
present, fix the type check in the iallocator request issuing code.

Signed-off-by: Gerard Oskamp <gjo@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

5 years agoImprove cfgupgrade output in case of errors
Hrvoje Ribicic [Wed, 28 Oct 2015 14:21:06 +0000 (15:21 +0100)]
Improve cfgupgrade output in case of errors

By logging with the exception function instead of the error function,
and showing the error content without the stack trace unless explicitly
debugging.

Signed-off-by: Gerard Oskamp <gjo@google.com>
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

5 years agoFix upgrades of instances with missing creation time
Hrvoje Ribicic [Tue, 27 Oct 2015 18:38:16 +0000 (18:38 +0000)]
Fix upgrades of instances with missing creation time

Some instances from very old Ganeti versions may not have any creation
time information embedded in the config. The upgrade code does not
expect this, and crashes horribly when trying to populate newly
separate disk objects with the same creation time, and this patch
fixes things by inserting a fake value: 0.

The value was chosen because the serialization and deserialization of
such an instance in Haskell yields a value of 0 for the ctime, making
the time consistent between instance and disk. While showing the epoch
time instead of N/A in gnt-instance info is suboptimal, due to the age
of the Ganeti version in which these instances must have been created,
they are at least still ordered correctly.

Signed-off-by: Gerard Oskamp <gjo@google.com>
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

5 years agoReduce flakyness of GetCmdline test on slow machines
Klaus Aehlig [Wed, 28 Oct 2015 10:54:18 +0000 (11:54 +0100)]
Reduce flakyness of GetCmdline test on slow machines

The GetCmdline test verifies that we can get the command line
of a running process via the procfs. To not have to care about
cleanup, the test creates an ephemeral process for this. While
two wall-clock seconds seem more than enough for a single read
from the procfs on nowadays machines, this is not true for some
of the public buildbot (virtual) machines which are extremely
low on resources and can have really heavy load; this causes
flakyness of that test there. Mitigate this by increasing the
life time of the process.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

5 years agoRemove duplicated words
Lisa Velden [Tue, 27 Oct 2015 15:43:13 +0000 (16:43 +0100)]
Remove duplicated words

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

5 years agoSupport force option for deactivate disks on RAPI
Klaus Aehlig [Tue, 27 Oct 2015 14:32:26 +0000 (15:32 +0100)]
Support force option for deactivate disks on RAPI

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

5 years agoMerge branch 'stable-2.14' into stable-2.15
Klaus Aehlig [Fri, 23 Oct 2015 11:32:06 +0000 (13:32 +0200)]
Merge branch 'stable-2.14' into stable-2.15

* stable-2.14
  (no changes)

* stable-2.13
  Renew-crypto: stop daemons on master node first
  Mention manual creation of {shared,}file paths in UPGRADE
  Don't warn about broken SSH setup of offline nodes

* stable-2.12
  Fix inconsistency in python and haskell objects
  Add notSerializeDefault default field option
  Move design-disks.rst to drafts

* stable-2.11
  Fix default for --default-iallocator-params

Conflicts:
doc/design-draft.rst
Resolution:
take all additions

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

5 years agoMerge branch 'stable-2.13' into stable-2.14
Klaus Aehlig [Fri, 23 Oct 2015 07:52:51 +0000 (09:52 +0200)]
Merge branch 'stable-2.13' into stable-2.14

* stable-2.13
  Renew-crypto: stop daemons on master node first
  Mention manual creation of {shared,}file paths in UPGRADE
  Don't warn about broken SSH setup of offline nodes

* stable-2.12
  Fix inconsistency in python and haskell objects
  Add notSerializeDefault default field option
  Move design-disks.rst to drafts

* stable-2.11
  Fix default for --default-iallocator-params

Conflicts:
src/Ganeti/THH.hs
Resolution:
take all additions

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

5 years agoMake htools tolerate missing "dtotal" and "dfree" on luxi
Klaus Aehlig [Tue, 16 Jun 2015 09:15:48 +0000 (11:15 +0200)]
Make htools tolerate missing "dtotal" and "dfree" on luxi

If a cluster allows sharedfile as only disk template, the amount of
total and free disk space might not be available. This is perfectly
normal, hence make the luxi backend handle it gracefully and just report
0 available disk on 0 total disk.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Cherry-picked-from: 49644203
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

5 years agoMerge branch 'stable-2.12' into stable-2.13
Klaus Aehlig [Thu, 22 Oct 2015 08:51:36 +0000 (10:51 +0200)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Fix inconsistency in python and haskell objects
  Add notSerializeDefault default field option
  Move design-disks.rst to drafts

* stable-2.11
  Fix default for --default-iallocator-params

Conflicts:
doc/design-draft.rst
doc/index.rst
lib/cli.py

Resolution:
for lib/cli.py follow the code move
for the rest, take all additions.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

5 years agoMerge branch 'stable-2.11' into stable-2.12
Klaus Aehlig [Thu, 22 Oct 2015 07:13:23 +0000 (09:13 +0200)]
Merge branch 'stable-2.11' into stable-2.12

* stable-2.11
  Fix default for --default-iallocator-params

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

5 years agoFix default for --default-iallocator-params
Klaus Aehlig [Wed, 21 Oct 2015 15:36:23 +0000 (17:36 +0200)]
Fix default for --default-iallocator-params

We need to distinguish between the option not being provided
(i.e., no change requested) and the option being empty (i.e.,
a request to reset the value). Therefore, use None as a default,
not {}.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

5 years agoRenew-crypto: stop daemons on master node first
Helga Velroyen [Wed, 21 Oct 2015 10:51:37 +0000 (12:51 +0200)]
Renew-crypto: stop daemons on master node first

Otherwise, this can create problems when restarting
the nodes due to voting issues.

Signed-off-by: Gerard Oskamp <gjo@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

5 years agoMention manual creation of {shared,}file paths in UPGRADE
Helga Velroyen [Thu, 15 Oct 2015 14:11:33 +0000 (16:11 +0200)]
Mention manual creation of {shared,}file paths in UPGRADE

This fixes Issue 653. It was unclear whether or not
'ensure-dirs' creates the directories for file and
sharedfile storage. This patch extends the documentation
to clarify this.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

5 years agoDon't warn about broken SSH setup of offline nodes
Helga Velroyen [Wed, 14 Oct 2015 08:24:33 +0000 (10:24 +0200)]
Don't warn about broken SSH setup of offline nodes

This fixes issue 1131. 'gnt-cluster verify' should stop
complaining about broken SSH setups of offline nodes.

Additionally, this fixes a problem when readding nodes.
In some cases, Ganeti complains about a possible attack,
which is a valid case for readding a node (if a key
renew took place between offlining and readding the node).

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

5 years agoFix inconsistency in python and haskell objects
Oleg Ponomarev [Mon, 12 Oct 2015 14:25:33 +0000 (16:25 +0200)]
Fix inconsistency in python and haskell objects

Currently hv/disk_state_static parameters are supported only for cluster
object properly. For node groups and nodes they were introduced in
2da9f556, however only on the python side. This could cause problems
during upgrades from old versions.

This patch adds hv and disk states fields to haskell objects as a
notSerializedDefaultField which will fix the problem without the changes
in behaviour. Also it modifies corresponding haskell arbitrary instances.

The patch is inspired by e78fb0d6 and 553363a3.

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

5 years agoAdd notSerializeDefault default field option
Oleg Ponomarev [Mon, 12 Oct 2015 14:25:32 +0000 (16:25 +0200)]
Add notSerializeDefault default field option

Default field with notSerializedDefault flag set is a default field which
will be serialized only if it's value differs from the default one. This
flag can be set by using notSerializedDefaultField field type instead of
defaultField field type.

This field is introduced in order to fix a bug of inconsistency between
haskell and python config modules which leads to inconsistent config
after ganeti updgrade.

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Cherry-picked from: c0a2c62b9ad96c3e35cae0ffdcdf63a09164f537

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

5 years agoFor queries, take the correct base address of an IP block
Klaus Aehlig [Fri, 9 Oct 2015 16:15:02 +0000 (18:15 +0200)]
For queries, take the correct base address of an IP block

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

5 years agoFix computation in network blocks
Klaus Aehlig [Fri, 9 Oct 2015 15:58:26 +0000 (17:58 +0200)]
Fix computation in network blocks

...by differentiating between the provided address and
the base address of the block. E.g., 10.0.0.1/29 and 10.0.0.0/29
contain the same IP addresses; in particular, the first address is
10.0.0.0.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>