ganeti-github.git
4 years agoRevision bump for 2.15.2 v2.15.2
Hrvoje Ribicic [Wed, 16 Dec 2015 12:16:57 +0000 (12:16 +0000)]
Revision bump for 2.15.2

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUpdate NEWS file for 2.15.2
Hrvoje Ribicic [Wed, 16 Dec 2015 12:16:39 +0000 (12:16 +0000)]
Update NEWS file for 2.15.2

With the security information and a list of minor changes.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMerge branch 'stable-2.14' into stable-2.15
Hrvoje Ribicic [Wed, 16 Dec 2015 11:09:38 +0000 (12:09 +0100)]
Merge branch 'stable-2.14' into stable-2.15

* stable-2.14
  Revision bump for 2.14.2
  Update NEWS file for 2.14.2

* stable-2.13
  Revision bump for 2.13.3
  Update NEWS file for 2.13.3

* stable-2.12
  Bump revision number for 2.12.6
  Update NEWS file for 2.12.6

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Merge entries
  configure.ac - Take 2.15 revision numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoRevision bump for 2.14.2 v2.14.2
Hrvoje Ribicic [Tue, 15 Dec 2015 17:54:17 +0000 (18:54 +0100)]
Revision bump for 2.14.2

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUpdate NEWS file for 2.14.2
Hrvoje Ribicic [Tue, 15 Dec 2015 17:53:11 +0000 (18:53 +0100)]
Update NEWS file for 2.14.2

With the security issues text and a list of minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMerge branch 'stable-2.13' into stable-2.14
Hrvoje Ribicic [Tue, 15 Dec 2015 14:44:16 +0000 (15:44 +0100)]
Merge branch 'stable-2.13' into stable-2.14

* stable-2.13
  Revision bump for 2.13.3
  Update NEWS file for 2.13.3

* stable-2.12
  Bump revision number for 2.12.6
  Update NEWS file for 2.12.6

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Merged entries
  configure.ac - Took 2.14 version numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoRevision bump for 2.13.3 v2.13.3
Hrvoje Ribicic [Mon, 14 Dec 2015 18:00:43 +0000 (19:00 +0100)]
Revision bump for 2.13.3

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUpdate NEWS file for 2.13.3
Hrvoje Ribicic [Mon, 14 Dec 2015 17:59:26 +0000 (18:59 +0100)]
Update NEWS file for 2.13.3

With the security issues text and a list of minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMerge branch 'stable-2.12' into stable-2.13
Hrvoje Ribicic [Mon, 14 Dec 2015 17:33:14 +0000 (18:33 +0100)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Bump revision number for 2.12.6
  Update NEWS file for 2.12.6

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Merge entries
  configure.ac - Take 2.13 version numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoBump revision number for 2.12.6 v2.12.6
Hrvoje Ribicic [Mon, 14 Dec 2015 16:42:03 +0000 (17:42 +0100)]
Bump revision number for 2.12.6

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoUpdate NEWS file for 2.12.6
Hrvoje Ribicic [Mon, 14 Dec 2015 16:41:09 +0000 (17:41 +0100)]
Update NEWS file for 2.12.6

With the security issues text and a list of minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.11' into stable-2.12
Hrvoje Ribicic [Mon, 14 Dec 2015 16:15:14 +0000 (17:15 +0100)]
Merge branch 'stable-2.11' into stable-2.12

* stable-2.11
  Revision bump for 2.11.8
  Update NEWS file for 2.11.8

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Merged entries
  configure.ac - Took 2.12 version numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoRevision bump for 2.11.8 v2.11.8
Hrvoje Ribicic [Mon, 14 Dec 2015 14:07:23 +0000 (15:07 +0100)]
Revision bump for 2.11.8

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoUpdate NEWS file for 2.11.8
Hrvoje Ribicic [Mon, 14 Dec 2015 14:06:50 +0000 (15:06 +0100)]
Update NEWS file for 2.11.8

With the security issues text and a list of minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.10' into stable-2.11
Hrvoje Ribicic [Mon, 14 Dec 2015 13:13:03 +0000 (14:13 +0100)]
Merge branch 'stable-2.10' into stable-2.11

* stable-2.10
  Version bump for 2.10.8
  Update NEWS file for 2.10.8

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - Combine NEWS entries from both versions
  configure.ac - Take correct version numbers

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoVersion bump for 2.10.8 v2.10.8
Hrvoje Ribicic [Fri, 11 Dec 2015 11:09:21 +0000 (12:09 +0100)]
Version bump for 2.10.8

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoUpdate NEWS file for 2.10.8
Hrvoje Ribicic [Fri, 11 Dec 2015 11:08:22 +0000 (12:08 +0100)]
Update NEWS file for 2.10.8

With the security issues text and list minor issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.9' into stable-2.10
Hrvoje Ribicic [Thu, 10 Dec 2015 18:04:48 +0000 (19:04 +0100)]
Merge branch 'stable-2.9' into stable-2.10

* stable-2.9
  Bump revision number
  Update NEWS file for 2.9.7 release
  Improve RAPI section on security

Conflicts:
  NEWS - leave 2.9.7 info in
  configure.ac - revert version bump

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoBump revision number stable-2.9 v2.9.7
Hrvoje Ribicic [Thu, 10 Dec 2015 16:40:51 +0000 (17:40 +0100)]
Bump revision number

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUpdate NEWS file for 2.9.7 release
Hrvoje Ribicic [Thu, 10 Dec 2015 16:39:53 +0000 (17:39 +0100)]
Update NEWS file for 2.9.7 release

... with security release info and minor changes.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoImprove RAPI section on security
Hrvoje Ribicic [Thu, 10 Dec 2015 13:22:01 +0000 (14:22 +0100)]
Improve RAPI section on security

The RAPI section on security has been improved with new information
related on how users can lock RAPI down as they see fit, and what are
the risks involved with default settings.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.14' into stable-2.15
Hrvoje Ribicic [Fri, 4 Dec 2015 15:06:50 +0000 (16:06 +0100)]
Merge branch 'stable-2.14' into stable-2.15

* stable-2.14
  Fix lines with more than 80 characters
  Add more detach/attach sequence tests
  Allow disk attachment to diskless instances
  Improve tests for attaching disks

* stable-2.13
  (no changes)

* stable-2.12
  Restrict showing of DRBD secret using types
  Calculate correct affected nodes set in InstanceChangeGroup

* stable-2.11
  (no changes)

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.13' into stable-2.14
Hrvoje Ribicic [Thu, 3 Dec 2015 22:55:20 +0000 (22:55 +0000)]
Merge branch 'stable-2.13' into stable-2.14

* stable-2.13
  (no changes)

* stable-2.12
  Restrict showing of DRBD secret using types
  Calculate correct affected nodes set in InstanceChangeGroup

* stable-2.11
  (no changes)

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  src/Ganeti/Objects.hs - Followed code to Disk.hs
  test/hs/Test/Ganeti/Objects.hs - Added Private to disk definition

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoMerge branch 'stable-2.12' into stable-2.13
Hrvoje Ribicic [Thu, 3 Dec 2015 21:13:39 +0000 (21:13 +0000)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Restrict showing of DRBD secret using types
  Calculate correct affected nodes set in InstanceChangeGroup

* stable-2.11
  (no changes)

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoRestrict showing of DRBD secret using types
Hrvoje Ribicic [Tue, 1 Dec 2015 16:11:38 +0000 (16:11 +0000)]
Restrict showing of DRBD secret using types

While the Python changes from 2.9 do prevent Ganeti from accidentally
revealing the Haskell secret, they may not do so forever. The queries
are planned to switch from Python to Haskell at some point, and should
someone want to use the DRBD secret, they can do so easily.

As a more elegant way of hiding the secret, wrap it in a Private
wrapper, preventing it from leaking out unless explicitly requested.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.11' into stable-2.12
Hrvoje Ribicic [Tue, 1 Dec 2015 15:57:49 +0000 (15:57 +0000)]
Merge branch 'stable-2.11' into stable-2.12

* stable-2.11
  (no changes)

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  lib/client/gnt_instance.py - taken the 2.11 version, with explicit
                               parameter use
  qa/qa_rapi.py - merged imports, resolved trivial conflict

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.10' into stable-2.11
Hrvoje Ribicic [Mon, 30 Nov 2015 16:12:42 +0000 (17:12 +0100)]
Merge branch 'stable-2.10' into stable-2.11

* stable-2.10
  (no changes)

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  qa/qa_rapi.py - simply append new changes

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.9' into stable-2.10
Hrvoje Ribicic [Mon, 30 Nov 2015 15:49:09 +0000 (16:49 +0100)]
Merge branch 'stable-2.9' into stable-2.10

* stable-2.9
  QA: Ensure the DRBD secret is not retrievable via RAPI
  Redact the DRBD secret in instance queries
  Do not attempt to use the DRBD secret in gnt-instance info

Conflicts:
  lib/cmdlib/instance_query.py - removed physical_id changes

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoQA: Ensure the DRBD secret is not retrievable via RAPI
Hrvoje Ribicic [Fri, 27 Nov 2015 17:32:42 +0000 (17:32 +0000)]
QA: Ensure the DRBD secret is not retrievable via RAPI

The best way to ensure that the DRBD secret does not inadvertently leak
is to introduce a QA test examining the output of the interface in
which the leak was originally introduced.

The test added determines the DRBD secret and makes RAPI requests,
examining them for its presence and failing if a match is found.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoRedact the DRBD secret in instance queries
Hrvoje Ribicic [Fri, 27 Nov 2015 15:58:13 +0000 (15:58 +0000)]
Redact the DRBD secret in instance queries

As the DRBD secret should be used only by Ganeti internals, replacing
the actual secret with None does not hamper Ganeti's work, while
preventing the secret from being leaked.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoDo not attempt to use the DRBD secret in gnt-instance info
Hrvoje Ribicic [Fri, 21 Aug 2015 19:46:18 +0000 (19:46 +0000)]
Do not attempt to use the DRBD secret in gnt-instance info

... so just redact what is output.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoFix lines with more than 80 characters
Lisa Velden [Fri, 27 Nov 2015 10:25:55 +0000 (11:25 +0100)]
Fix lines with more than 80 characters

Previous refactoring has introduced lines with too many characters.
This patch fixes this.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoAdd more detach/attach sequence tests
Lisa Velden [Wed, 25 Nov 2015 16:57:18 +0000 (17:57 +0100)]
Add more detach/attach sequence tests

Test detach/attach sequences with an instance that becomes diskless
after detaching its disk and also test detach/attach with drbd disks.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoAllow disk attachment to diskless instances
Lisa Velden [Wed, 25 Nov 2015 15:00:45 +0000 (16:00 +0100)]
Allow disk attachment to diskless instances

As only DRBD disks can be associated to more nodes than the instance
where we want to attach the disk to, we have to change the check for
associated nodes, too.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoImprove tests for attaching disks
Lisa Velden [Wed, 25 Nov 2015 13:53:39 +0000 (14:53 +0100)]
Improve tests for attaching disks

by associating disks and instances to a specific node.
Also refactor mock uuids and mock disk names into variables.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoCompute lock allocation strictly
Klaus Aehlig [Thu, 26 Nov 2015 16:49:38 +0000 (17:49 +0100)]
Compute lock allocation strictly

Given that on updates it has to be fully computed anyway, do not
accumulate thunks during the computation.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoCalculate correct affected nodes set in InstanceChangeGroup
Oleg Ponomarev [Fri, 20 Nov 2015 20:45:11 +0000 (21:45 +0100)]
Calculate correct affected nodes set in InstanceChangeGroup

This is the fix for the issue 1144. The nodes affected by the
InstanceChangeGroup logical unit were calculated incorrectly and that
broke 'gnt-instance change-group --to' operation. This patch fixes it.

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoDocument the decission why optimisation is turned off
Klaus Aehlig [Thu, 19 Nov 2015 13:27:03 +0000 (14:27 +0100)]
Document the decission why optimisation is turned off

Commit c22a35 removed an argument of readJSONWithDesc which
caused some versions of ghc to go too crazy in optimising,
so it had to be turned off for some files. Document that reason
in a comment.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoDon't keep input for error messages
Klaus Aehlig [Wed, 18 Nov 2015 13:59:36 +0000 (14:59 +0100)]
Don't keep input for error messages

When generating error messages, the raw JSValue is rarely
useful. However, keeping it for error messages---even if
only in the unused branch of an if statement---prevents this
value from going out of scope.

Note: with the smaller number of arguments in the readJSONWithDesc
function, newer versions of ghc try too fancy optimisations and thus
run out of memory; hence we have to reduce the ghc optimisation level
for some files.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUse dict.copy instead of deepcopy
Helga Velroyen [Wed, 18 Nov 2015 08:44:43 +0000 (09:44 +0100)]
Use dict.copy instead of deepcopy

Due to a bug in python, deepcopy does not work on
the dictionaries we use for SSH updates. This patch
replaces the use of deepcopy by the built-in copy
function of dictionaries.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoUse bulk-adding of keys in renew-crypto
Helga Velroyen [Thu, 12 Nov 2015 12:48:59 +0000 (13:48 +0100)]
Use bulk-adding of keys in renew-crypto

This patch makes renew-crypto actually use the bulk-adding
function of SSH keys rather than adding each key
individually. This patch also adds a unit tests where the
bulk-adding is tested with a diverse set of keys (master
candidates, potential master candidates, normal nodes).

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMake NodeSshKeyAdd use its *Bulk companion
Helga Velroyen [Thu, 12 Nov 2015 10:21:45 +0000 (11:21 +0100)]
Make NodeSshKeyAdd use its *Bulk companion

Since the bulk-version of adding keys is subsuming the
functionality of adding a single key, this patch makes
NodeSshKeyAdd internally use the *Bulk version. The
unit tests in place make sure no functionality is
changed.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoUnit test bulk-adding normal nodes
Helga Velroyen [Thu, 12 Nov 2015 10:06:26 +0000 (11:06 +0100)]
Unit test bulk-adding normal nodes

This patch adds a unit test that tests adding a bulk
of normal nodes' SSH keys to the cluster.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoUnit test for bulk-adding pot. master candidates
Helga Velroyen [Thu, 12 Nov 2015 10:05:15 +0000 (11:05 +0100)]
Unit test for bulk-adding pot. master candidates

This patch adds a unit tests for bulk-adding SSH keys
of potential master candidates.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoIntroduce bulk-adding of SSH keys
Helga Velroyen [Wed, 11 Nov 2015 15:54:31 +0000 (16:54 +0100)]
Introduce bulk-adding of SSH keys

This patch introduces a backend function to add a set of
SSH keys to the nodes (rather than one key at a time).
The bulk-adding function is having the same structure
as the original one, but is adapted to work with a set
of keys rather than one key.

This patch also adds a unit test for testing the
bulk-adding of keys.

Note that this patch only adds the bulk-adding function
but does not use it yet. In the following patches of
this series, we will add more unit tests and at the
end integrate the bulk-adding function into
renew-crypto.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoPause watcher during performance QA
Klaus Aehlig [Tue, 17 Nov 2015 14:16:28 +0000 (15:16 +0100)]
Pause watcher during performance QA

Our performance QA tests are intended to alert us if some common
task suddenly takes longer. To serve this purpose, they need to provide
reproducible results. Hence avoid any interference with watcher-submitted
jobs by pausing the watcher during performance QA tests.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoSend answers strictly
Klaus Aehlig [Mon, 16 Nov 2015 14:05:45 +0000 (15:05 +0100)]
Send answers strictly

When sending an answer over a domain socket, the recipient
won't process that answer anyway before it is complete. So
we can as well assemble one ByteString first and send it over
the wire all at once, thus saving a few system calls.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoStore keys as ByteStrings
Klaus Aehlig [Thu, 12 Nov 2015 13:51:16 +0000 (14:51 +0100)]
Store keys as ByteStrings

Keys to maps are only used to look up values, so
a compact representation does impact flexibility.
However, it does save on memory usage; having more
locality in the keys also improves time when comparing
them.

While there, also refrain from linearly looking through
keys searching for partial matches where partial matches
are not desired (e.g., when looking up things by uuid).

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoMerge branch 'stable-2.14' into stable-2.15
Klaus Aehlig [Thu, 12 Nov 2015 10:45:16 +0000 (11:45 +0100)]
Merge branch 'stable-2.14' into stable-2.15

* stable-2.14
  Fix faulty iallocator type check
  Improve cfgupgrade output in case of errors

* stable-2.13
  Extend timeout for gnt-cluster renew-crypto
  Reduce flakyness of GetCmdline test on slow machines
  Remove duplicated words

* stable-2.12
  Revert "Also consider connection time out a network error"
  Clone lists before modifying
  Make lockConfig call retryable
  Return the correct error code in the post-upgrade script
  Make openssl refrain from DH altogether
  Fix upgrades of instances with missing creation time

* stable-2.11
  (no changes)

* stable-2.10
  Remove -X from hspace man page
  Make htools tolerate missing "dtotal" and "dfree" on luxi

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoEncode UUIDs as ByteStrings
Klaus Aehlig [Wed, 11 Nov 2015 11:07:03 +0000 (12:07 +0100)]
Encode UUIDs as ByteStrings

UUIDs are fixed-length strings at which we either look
completely or not at all. Moreover, we do not do any
computations on them. Therefore, we can chose a more
compact representation on them, resulting in reduced memory
foot print.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoPrefer the UuidObject type class over specific functions
Klaus Aehlig [Wed, 11 Nov 2015 11:26:24 +0000 (12:26 +0100)]
Prefer the UuidObject type class over specific functions

The UuidObject type class provides a clean interface to
obtain the UUID of an object. Prefer this interface over
hard-coding the specific functions all over the place.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoMerge branch 'stable-2.13' into stable-2.14
Oleg Ponomarev [Wed, 11 Nov 2015 18:01:36 +0000 (19:01 +0100)]
Merge branch 'stable-2.13' into stable-2.14

* stable-2.13
  Extend timeout for gnt-cluster renew-crypto

* stable-2.12
  Revert "Also consider connection time out a network error"
  Clone lists before modifying
  Make lockConfig call retryable

* stable-2.11
  (no changes)

* stable-2.10
  Remove -X from hspace man page
  Make htools tolerate missing "dtotal" and "dfree" on luxi

Conflicts:
    tools/cfgupgrade
Resolution
    take the change into lib/tools/cfgupgrade

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.12' into stable-2.13
Oleg Ponomarev [Wed, 11 Nov 2015 17:14:51 +0000 (18:14 +0100)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Revert "Also consider connection time out a network error"
  Clone lists before modifying
  Make lockConfig call retryable

* stable-2.11
  (no changes)

* stable-2.10
  Remove -X from hspace man page
  Make htools tolerate missing "dtotal" and "dfree" on luxi

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.11' into stable-2.12
Oleg Ponomarev [Wed, 11 Nov 2015 16:04:40 +0000 (17:04 +0100)]
Merge branch 'stable-2.11' into stable-2.12

    * stable-2.11
      (no changes)

    * stable-2.10
      Remove -X from hspace man page
      Make htools tolerate missing "dtotal" and "dfree" on luxi

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Reviewed-by: Liza Velden <velden@google.com>

4 years agoMerge branch 'stable-2.10' into stable-2.11
Klaus Aehlig [Wed, 11 Nov 2015 15:51:42 +0000 (16:51 +0100)]
Merge branch 'stable-2.10' into stable-2.11

* stable-2.10
  Remove -X from hspace man page
  Make htools tolerate missing "dtotal" and "dfree" on luxi

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoRevert "Also consider connection time out a network error"
Klaus Aehlig [Tue, 10 Nov 2015 16:47:44 +0000 (17:47 +0100)]
Revert "Also consider connection time out a network error"

This reverts commit 84c17185ad47070944c64ab64a8c7dfd60a260f9.
We use RetryOnNetworkError for basically every form of internal
communication. While it makes sense to retry---given that we
assume daemons might come and go at any time---we can only do
so safely, if we positively know that we did not cause any
side effect. Given that not all our requests are idempotent
(e.g., submitting jobs is not)---in fact, the majority is
not--, retrying on timeouts is not safe.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoClone lists before modifying
Klaus Aehlig [Tue, 10 Nov 2015 15:40:47 +0000 (16:40 +0100)]
Clone lists before modifying

When an opcode expands to a list of jobs, we extend the reason trail
of the new jobs with that of the original opcode that expanded to them.
Before modifying the reason trail, however, we should duplicate it to
avoid side effects on shared copies---like the default empty list.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoAssign the variables before use (bugfix for dee6adb9)
Oleg Ponomarev [Mon, 9 Nov 2015 16:28:38 +0000 (17:28 +0100)]
Assign the variables before use (bugfix for dee6adb9)

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoExtend QA to detect autopromotion errors
Helga Velroyen [Fri, 6 Nov 2015 10:27:41 +0000 (11:27 +0100)]
Extend QA to detect autopromotion errors

The issue that was fixed with the previous patch would
have been detected earlier if the QA would actually
run a 'verify' after the modify operations. For 'verify'
to not raise false negatives, we need to first reduce
the candidate pool size, because otherwise QA fails
with a warning about the mininmum pool size being
violated.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoHandle SSH key distribution on auto promotion
Helga Velroyen [Fri, 6 Nov 2015 10:26:08 +0000 (11:26 +0100)]
Handle SSH key distribution on auto promotion

This fixes the missing SSH key distribution in case
a node gets autopromoted to master candidate.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoDo not remove authorized key of node itself
Helga Velroyen [Fri, 6 Nov 2015 09:11:19 +0000 (10:11 +0100)]
Do not remove authorized key of node itself

This fixes a small bug that if a node was demoted
from master candidate, that its own public key
was removed from its own authorized key file.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoFix indentation
Lisa Velden [Thu, 5 Nov 2015 10:36:57 +0000 (11:36 +0100)]
Fix indentation

so that the method can be called correctly.

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMake lockConfig call retryable
Klaus Aehlig [Wed, 4 Nov 2015 13:52:16 +0000 (14:52 +0100)]
Make lockConfig call retryable

Locking the configuration is naturally idempotent. However,
the corresponding WConfD call had a check refusing to lock
the config, if the caller has already locked it, arguing that
this should not happen. That argument misses that we have the
built-in assumption that daemons might be restarted at any time,
including the moment where a request is processed, but the caller
did not get the answer yet. So allow retries, hower logging that
they occurred (as this should only happen rarely).

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoExtend timeout for gnt-cluster renew-crypto
Hrvoje Ribicic [Wed, 4 Nov 2015 13:01:38 +0000 (14:01 +0100)]
Extend timeout for gnt-cluster renew-crypto

With particularly large clusters, the renewal of SSH keys happening in
renew-crypto can take a long time to complete. While this should be
improved, an additional problem is that the RPC doing most of the work
has a default one-hour timeout. Given that it is preferable that the
operation completes, this patch bumps the timeout to four hours, which
should suffice even for 80+ node clusters.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMerge branch 'stable-2.12' into stable-2.13
Hrvoje Ribicic [Mon, 2 Nov 2015 17:49:36 +0000 (17:49 +0000)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Return the correct error code in the post-upgrade script
  Make openssl refrain from DH altogether
  Fix upgrades of instances with missing creation time

Conflicts:
cfgupgrade_unittest.py: merge version tests
tools/post-upgrade: return the correct error code for SSH
                    renewal as well

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoReturn the correct error code in the post-upgrade script
Hrvoje Ribicic [Mon, 2 Nov 2015 17:19:22 +0000 (17:19 +0000)]
Return the correct error code in the post-upgrade script

While we want all the post-upgrade actions to be undertaken, should one
of these fail, the correct error code should be returned so that the
upgrade script can report issues.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMake openssl refrain from DH altogether
Klaus Aehlig [Mon, 2 Nov 2015 10:44:34 +0000 (11:44 +0100)]
Make openssl refrain from DH altogether

As various ssl implementations have different ideas about
which dh key lengths are acceptable, refrain from standard
dh altogether (and not only from anonymous dh) to avoid
handshake problems.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoRemove -X from hspace man page
Klaus Aehlig [Mon, 26 Oct 2015 12:34:17 +0000 (13:34 +0100)]
Remove -X from hspace man page

hspace never had such an option.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

Cherry-picked-from: fa36daf4
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoFix faulty iallocator type check
Hrvoje Ribicic [Wed, 28 Oct 2015 17:56:23 +0000 (17:56 +0000)]
Fix faulty iallocator type check

Because the ignore-soft-errors parameter is optional rather than always
present, fix the type check in the iallocator request issuing code.

Signed-off-by: Gerard Oskamp <gjo@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoImprove cfgupgrade output in case of errors
Hrvoje Ribicic [Wed, 28 Oct 2015 14:21:06 +0000 (15:21 +0100)]
Improve cfgupgrade output in case of errors

By logging with the exception function instead of the error function,
and showing the error content without the stack trace unless explicitly
debugging.

Signed-off-by: Gerard Oskamp <gjo@google.com>
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoFix upgrades of instances with missing creation time
Hrvoje Ribicic [Tue, 27 Oct 2015 18:38:16 +0000 (18:38 +0000)]
Fix upgrades of instances with missing creation time

Some instances from very old Ganeti versions may not have any creation
time information embedded in the config. The upgrade code does not
expect this, and crashes horribly when trying to populate newly
separate disk objects with the same creation time, and this patch
fixes things by inserting a fake value: 0.

The value was chosen because the serialization and deserialization of
such an instance in Haskell yields a value of 0 for the ctime, making
the time consistent between instance and disk. While showing the epoch
time instead of N/A in gnt-instance info is suboptimal, due to the age
of the Ganeti version in which these instances must have been created,
they are at least still ordered correctly.

Signed-off-by: Gerard Oskamp <gjo@google.com>
Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoReduce flakyness of GetCmdline test on slow machines
Klaus Aehlig [Wed, 28 Oct 2015 10:54:18 +0000 (11:54 +0100)]
Reduce flakyness of GetCmdline test on slow machines

The GetCmdline test verifies that we can get the command line
of a running process via the procfs. To not have to care about
cleanup, the test creates an ephemeral process for this. While
two wall-clock seconds seem more than enough for a single read
from the procfs on nowadays machines, this is not true for some
of the public buildbot (virtual) machines which are extremely
low on resources and can have really heavy load; this causes
flakyness of that test there. Mitigate this by increasing the
life time of the process.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoRemove duplicated words
Lisa Velden [Tue, 27 Oct 2015 15:43:13 +0000 (16:43 +0100)]
Remove duplicated words

Signed-off-by: Lisa Velden <velden@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoSupport force option for deactivate disks on RAPI
Klaus Aehlig [Tue, 27 Oct 2015 14:32:26 +0000 (15:32 +0100)]
Support force option for deactivate disks on RAPI

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoMerge branch 'stable-2.14' into stable-2.15
Klaus Aehlig [Fri, 23 Oct 2015 11:32:06 +0000 (13:32 +0200)]
Merge branch 'stable-2.14' into stable-2.15

* stable-2.14
  (no changes)

* stable-2.13
  Renew-crypto: stop daemons on master node first
  Mention manual creation of {shared,}file paths in UPGRADE
  Don't warn about broken SSH setup of offline nodes

* stable-2.12
  Fix inconsistency in python and haskell objects
  Add notSerializeDefault default field option
  Move design-disks.rst to drafts

* stable-2.11
  Fix default for --default-iallocator-params

Conflicts:
doc/design-draft.rst
Resolution:
take all additions

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.13' into stable-2.14
Klaus Aehlig [Fri, 23 Oct 2015 07:52:51 +0000 (09:52 +0200)]
Merge branch 'stable-2.13' into stable-2.14

* stable-2.13
  Renew-crypto: stop daemons on master node first
  Mention manual creation of {shared,}file paths in UPGRADE
  Don't warn about broken SSH setup of offline nodes

* stable-2.12
  Fix inconsistency in python and haskell objects
  Add notSerializeDefault default field option
  Move design-disks.rst to drafts

* stable-2.11
  Fix default for --default-iallocator-params

Conflicts:
src/Ganeti/THH.hs
Resolution:
take all additions

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMake htools tolerate missing "dtotal" and "dfree" on luxi
Klaus Aehlig [Tue, 16 Jun 2015 09:15:48 +0000 (11:15 +0200)]
Make htools tolerate missing "dtotal" and "dfree" on luxi

If a cluster allows sharedfile as only disk template, the amount of
total and free disk space might not be available. This is perfectly
normal, hence make the luxi backend handle it gracefully and just report
0 available disk on 0 total disk.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>

Cherry-picked-from: 49644203
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoMerge branch 'stable-2.12' into stable-2.13
Klaus Aehlig [Thu, 22 Oct 2015 08:51:36 +0000 (10:51 +0200)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Fix inconsistency in python and haskell objects
  Add notSerializeDefault default field option
  Move design-disks.rst to drafts

* stable-2.11
  Fix default for --default-iallocator-params

Conflicts:
doc/design-draft.rst
doc/index.rst
lib/cli.py

Resolution:
for lib/cli.py follow the code move
for the rest, take all additions.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoMerge branch 'stable-2.11' into stable-2.12
Klaus Aehlig [Thu, 22 Oct 2015 07:13:23 +0000 (09:13 +0200)]
Merge branch 'stable-2.11' into stable-2.12

* stable-2.11
  Fix default for --default-iallocator-params

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoFix default for --default-iallocator-params
Klaus Aehlig [Wed, 21 Oct 2015 15:36:23 +0000 (17:36 +0200)]
Fix default for --default-iallocator-params

We need to distinguish between the option not being provided
(i.e., no change requested) and the option being empty (i.e.,
a request to reset the value). Therefore, use None as a default,
not {}.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>

4 years agoRenew-crypto: stop daemons on master node first
Helga Velroyen [Wed, 21 Oct 2015 10:51:37 +0000 (12:51 +0200)]
Renew-crypto: stop daemons on master node first

Otherwise, this can create problems when restarting
the nodes due to voting issues.

Signed-off-by: Gerard Oskamp <gjo@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

4 years agoMention manual creation of {shared,}file paths in UPGRADE
Helga Velroyen [Thu, 15 Oct 2015 14:11:33 +0000 (16:11 +0200)]
Mention manual creation of {shared,}file paths in UPGRADE

This fixes Issue 653. It was unclear whether or not
'ensure-dirs' creates the directories for file and
sharedfile storage. This patch extends the documentation
to clarify this.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Oleg Ponomarev <oponomarev@google.com>

4 years agoDon't warn about broken SSH setup of offline nodes
Helga Velroyen [Wed, 14 Oct 2015 08:24:33 +0000 (10:24 +0200)]
Don't warn about broken SSH setup of offline nodes

This fixes issue 1131. 'gnt-cluster verify' should stop
complaining about broken SSH setups of offline nodes.

Additionally, this fixes a problem when readding nodes.
In some cases, Ganeti complains about a possible attack,
which is a valid case for readding a node (if a key
renew took place between offlining and readding the node).

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoFix inconsistency in python and haskell objects
Oleg Ponomarev [Mon, 12 Oct 2015 14:25:33 +0000 (16:25 +0200)]
Fix inconsistency in python and haskell objects

Currently hv/disk_state_static parameters are supported only for cluster
object properly. For node groups and nodes they were introduced in
2da9f556, however only on the python side. This could cause problems
during upgrades from old versions.

This patch adds hv and disk states fields to haskell objects as a
notSerializedDefaultField which will fix the problem without the changes
in behaviour. Also it modifies corresponding haskell arbitrary instances.

The patch is inspired by e78fb0d6 and 553363a3.

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoAdd notSerializeDefault default field option
Oleg Ponomarev [Mon, 12 Oct 2015 14:25:32 +0000 (16:25 +0200)]
Add notSerializeDefault default field option

Default field with notSerializedDefault flag set is a default field which
will be serialized only if it's value differs from the default one. This
flag can be set by using notSerializedDefaultField field type instead of
defaultField field type.

This field is introduced in order to fix a bug of inconsistency between
haskell and python config modules which leads to inconsistent config
after ganeti updgrade.

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Cherry-picked from: c0a2c62b9ad96c3e35cae0ffdcdf63a09164f537

Signed-off-by: Oleg Ponomarev <oponomarev@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoFor queries, take the correct base address of an IP block
Klaus Aehlig [Fri, 9 Oct 2015 16:15:02 +0000 (18:15 +0200)]
For queries, take the correct base address of an IP block

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoFix computation in network blocks
Klaus Aehlig [Fri, 9 Oct 2015 15:58:26 +0000 (17:58 +0200)]
Fix computation in network blocks

...by differentiating between the provided address and
the base address of the block. E.g., 10.0.0.1/29 and 10.0.0.0/29
contain the same IP addresses; in particular, the first address is
10.0.0.0.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.14' into stable-2.15
Hrvoje Ribicic [Mon, 12 Oct 2015 14:01:02 +0000 (16:01 +0200)]
Merge branch 'stable-2.14' into stable-2.15

* stable-2.14
  Add test for tags accounting in hail
  Set node tags in iallocator htools backend

* stable-2.13
  Improve xl socat migrations

* stable-2.12
  QA: Retrieve only the RAPI certificate
  QA: Allow usage of specific RAPI certificates and files
  QA: Reload certificates only when renew-crypto has been run
  QA: Restart Ganeti after adding the RAPI users file
  QA: Add reading the RAPI password from a file
  QA: Allow the RAPI user to be set
  QA: Do not remove nodes from cluster without destroying it
  QA: Refactor RAPI handling
  Increase default disk size of burnin to 1G
  break line with more than 80 characters
  Only search for Python-2 interpreters
  Fix faulty comments / indentation
  Handle Xen 4.3 states better

* stable-2.11
  (no changes)

* stable-2.10
  Add a test for parsing of admin_state in IAlloc backend
  At IAlloc backend guess state from admin state

* stable-2.9
  Update harep's man page to notify users of its limitations

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

4 years agoMerge branch 'stable-2.13' into stable-2.14
Klaus Aehlig [Mon, 12 Oct 2015 12:54:00 +0000 (14:54 +0200)]
Merge branch 'stable-2.13' into stable-2.14

* stable-2.13
  Improve xl socat migrations

* stable-2.12
  QA: Retrieve only the RAPI certificate
  QA: Allow usage of specific RAPI certificates and files
  QA: Reload certificates only when renew-crypto has been run
  QA: Restart Ganeti after adding the RAPI users file
  QA: Add reading the RAPI password from a file
  QA: Allow the RAPI user to be set
  QA: Do not remove nodes from cluster without destroying it
  QA: Refactor RAPI handling
  Increase default disk size of burnin to 1G
  break line with more than 80 characters
  Only search for Python-2 interpreters
  Fix faulty comments / indentation
  Handle Xen 4.3 states better

* stable-2.11
  (no changes)

* stable-2.10
  Add a test for parsing of admin_state in IAlloc backend
  At IAlloc backend guess state from admin state

* stable-2.9
  Update harep's man page to notify users of its limitations

Conflicts:
src/Ganeti/HTools/Backend/IAlloc.hs
Resolution:
manually apply 9c1704a5 to stable-2.13

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMove design-disks.rst to drafts
Klaus Aehlig [Mon, 12 Oct 2015 12:15:07 +0000 (14:15 +0200)]
Move design-disks.rst to drafts

When, in commit 2676f31, the design for stand-alone disks
was added, it was not added to the list of draft designs,
but accidentally to the list of designs not shown in the index;
the latter, however, is only for implemented designs. As this
design still isn't fully implemented, fix this now.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.12' into stable-2.13
Klaus Aehlig [Fri, 9 Oct 2015 09:04:28 +0000 (11:04 +0200)]
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  QA: Retrieve only the RAPI certificate
  QA: Allow usage of specific RAPI certificates and files
  QA: Reload certificates only when renew-crypto has been run
  QA: Restart Ganeti after adding the RAPI users file
  QA: Add reading the RAPI password from a file
  QA: Allow the RAPI user to be set
  QA: Do not remove nodes from cluster without destroying it
  QA: Refactor RAPI handling
  Increase default disk size of burnin to 1G
  break line with more than 80 characters
  Only search for Python-2 interpreters
  Fix faulty comments / indentation
  Handle Xen 4.3 states better

* stable-2.11
  (no changes)

* stable-2.10
  Add a test for parsing of admin_state in IAlloc backend
  At IAlloc backend guess state from admin state

* stable-2.9
  Update harep's man page to notify users of its limitations

Conflicts:
qa/qa_cluster.py: trivial
qa/rapi-workload.py: keep removed (see c0065e0fa1730a477)

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.11' into stable-2.12
Klaus Aehlig [Thu, 8 Oct 2015 14:35:35 +0000 (16:35 +0200)]
Merge branch 'stable-2.11' into stable-2.12

* stable-2.11
  (no changes)

* stable-2.10
  Add a test for parsing of admin_state in IAlloc backend
  At IAlloc backend guess state from admin state

* stable-2.9
  Update harep's man page to notify users of its limitations

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoMerge branch 'stable-2.10' into stable-2.11
Klaus Aehlig [Thu, 8 Oct 2015 14:16:53 +0000 (16:16 +0200)]
Merge branch 'stable-2.10' into stable-2.11

* stable-2.10
  Add a test for parsing of admin_state in IAlloc backend
  At IAlloc backend guess state from admin state

* stable-2.9
  Update harep's man page to notify users of its limitations

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoQA: Retrieve only the RAPI certificate
Hrvoje Ribicic [Sun, 27 Sep 2015 21:55:51 +0000 (21:55 +0000)]
QA: Retrieve only the RAPI certificate

The QA previously took in the entire certificate file, along with the
private key. As this is really not necessary, change it to be more
conservative.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoQA: Allow usage of specific RAPI certificates and files
Hrvoje Ribicic [Wed, 23 Sep 2015 14:38:50 +0000 (16:38 +0200)]
QA: Allow usage of specific RAPI certificates and files

In some situations, we want to make sure the QA runs with a certain set
of certificates, secrets, users, and the like. This patch allows the QA
to look for a directory on the master node where all of these can be
found, and transplant them into the right place. This allow cluster
creation, renew-crypto, or any other cert-affecting operation to be
tested while preserving RAPI access.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoQA: Reload certificates only when renew-crypto has been run
Hrvoje Ribicic [Thu, 24 Sep 2015 10:36:31 +0000 (12:36 +0200)]
QA: Reload certificates only when renew-crypto has been run

When the cluster refreshes the RAPI certificate as it does in the
renew-crypto test, the stored certificate in the curl config of the
RAPI client has to be renewed. But it should only be renewed when the
test is enabled, so this patch moves that code into the test.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoQA: Restart Ganeti after adding the RAPI users file
Hrvoje Ribicic [Thu, 24 Sep 2015 21:20:30 +0000 (23:20 +0200)]
QA: Restart Ganeti after adding the RAPI users file

... otherwise we have no guarantee that the RAPI daemon will pick up
the change.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoQA: Add reading the RAPI password from a file
Hrvoje Ribicic [Tue, 22 Sep 2015 17:14:50 +0000 (19:14 +0200)]
QA: Add reading the RAPI password from a file

For situations where we're running the QA against a cluster which uses
a hashed password for access, it can be useful to be able to read the
password from a local file. This patch allows this to happen, throwing
in a few refactorings along the way.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoQA: Allow the RAPI user to be set
Hrvoje Ribicic [Tue, 22 Sep 2015 17:09:06 +0000 (19:09 +0200)]
QA: Allow the RAPI user to be set

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>

4 years agoQA: Do not remove nodes from cluster without destroying it
Hrvoje Ribicic [Tue, 22 Sep 2015 15:20:46 +0000 (17:20 +0200)]
QA: Do not remove nodes from cluster without destroying it

The Ganeti QA can be set up to optionally both create and destroy a
cluster during its runtime. Before this patch, the QA removed all the
nodes barring the master one at the end of a QA, regardless of whether
the cluster was supposed to be disassembled. This patch fixes this
behaviour and lets created clusters remain in place after a QA.

Signed-off-by: Hrvoje Ribicic <riba@google.com>
Reviewed-by: Lisa Velden <velden@google.com>