ganeti-github.git
8 years agoAdd "Attoparsec" to the optional haskell packages
Michele Tartara [Fri, 9 Nov 2012 15:46:05 +0000 (16:46 +0100)]
Add "Attoparsec" to the optional haskell packages

This will be needed for the data collectors of the monitoring agent.

* Detection of the library
* Creation of the appropriate variables
* Update to the installation documentation

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoDocumentation for the NODE_RES level
Helga Velroyen [Fri, 9 Nov 2012 13:52:50 +0000 (14:52 +0100)]
Documentation for the NODE_RES level

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoImprove make regen-vcs-version
Iustin Pop [Mon, 12 Nov 2012 16:35:53 +0000 (17:35 +0100)]
Improve make regen-vcs-version

"make regen-vcs-version" is run at every ./devel/upload time, in order
to have reasonably up-to-date information in the uploaded
scripts/binaries. However, this means it currently triggers
recompilation of (at least) Ganeti/Version.hs, and sometimes more than
that.

Since we only care about the contents of the file and not the
timestamp, let's change the make rule so that it only updates the file
if there are actually changes in the content, as checked by
"cmp". This results in a much faster ./devel/upload, especially for
small changes in the (non-Haskell) code, or when the code has already
been compiled.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoFix two hlint warnings
Iustin Pop [Mon, 12 Nov 2012 16:35:32 +0000 (17:35 +0100)]
Fix two hlint warnings

Sorry, I broke lint again :), by introducing two sub-standard changes.

Additionally, this silences an older existing warning that only
triggers with some versions of hlint (e.g. 1.8.28 which is present in
Wheezy).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoRunCmd: Expose "postfork" callback
Michael Hanselmann [Fri, 9 Nov 2012 15:12:22 +0000 (16:12 +0100)]
RunCmd: Expose "postfork" callback

The “_postfork_fn” parameter was only used for tests until now. To
implement a good locking scheme, remote commands must also make use of
this callback to release a lock when the command was successfully
started (but did not yet finish).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoMerge branch 'devel-2.6' into master
Iustin Pop [Mon, 12 Nov 2012 16:11:17 +0000 (17:11 +0100)]
Merge branch 'devel-2.6' into master

* devel-2.6:
  Improve error message when migration status fail
  Fix type error in kvm/GetMigrationStatus
  Fix PID file writing in Haskell daemons

Conflicts (both trivial):
        htools/Ganeti/Daemon.hs (_writePidFile rename)
        lib/hypervisor/hv_kvm.py (master fixed an indentation issue)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoMerge branch 'stable-2.6' into devel-2.6
Iustin Pop [Mon, 12 Nov 2012 14:13:39 +0000 (15:13 +0100)]
Merge branch 'stable-2.6' into devel-2.6

* stable-2.6:
  Improve error message when migration status fail
  Fix type error in kvm/GetMigrationStatus
  Fix PID file writing in Haskell daemons

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoImprove error message when migration status fail
Iustin Pop [Mon, 12 Nov 2012 11:57:58 +0000 (12:57 +0100)]
Improve error message when migration status fail

Commit 6a1434d7 (“Make migration RPC non-blocking”) changed the API
for reporting migration status, but has a small cosmetic bug: if the
migration status if failure, but the RPC itself to get the status
didn't fail, it shows the following error message:

  Could not migrate instance instance2: None

since it always uses result.fail_msg, irrespective of which part of
the if condition failed.

This patch simply updates the msg if not already set, leading to:

  Could not migrate instance instance2: hypervisor returned failure

Proper error display can be done once the migration status objects can
return failure information as well, beside status.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

8 years agoFix type error in kvm/GetMigrationStatus
Iustin Pop [Mon, 12 Nov 2012 11:50:52 +0000 (12:50 +0100)]
Fix type error in kvm/GetMigrationStatus

Commit 6a1434d7 (“Make migration RPC non-blocking”) changed from
raising HypervisorErrors to returning MigrationStatus
objects. However, these objects don't have an "info" attribute, so
they can't pass a reason back (which is in itself a bug); but the KVM
hypervisor code attempts to do so, and fails at runtime with:

  Failed to get migration status: 'MigrationStatus' object has no attribute 'info'

instead of the intended:

  Migration failed, aborting: too many broken 'info migrate' answers

For now (on stable-2.6), let's just remove the "info" reason, and
later we can add it back properly once we have a way to correctly
represent migration status failures in the LU.

This fixes issue 297.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoFix PID file writing in Haskell daemons
Iustin Pop [Mon, 12 Nov 2012 11:14:37 +0000 (12:14 +0100)]
Fix PID file writing in Haskell daemons

Currently, the code uses createFile, which has the effect of always
truncating the file. This is bad, as the content of the PID file is
wiped even when we wouldn't be able to lock it!

We switch to openFd (createFile is just a wrapper over that), and we
use an explicit set of flags; defaultFileFlags is already safe
(trunc=False), but I prefer to set it explicitly with our desired
flags.

Note that this bug doesn't manifest in normal usage, as daemon-util
won't try to start the daemon if already running. But if anyone or
anything does call ganeti-confd explicitly, the pid file will be
emptied and the daemon will keep trying to be restarted forever…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoRemove unsafePerformIO usage
Iustin Pop [Thu, 8 Nov 2012 17:52:31 +0000 (18:52 +0100)]
Remove unsafePerformIO usage

We need to change a few things, most importantly CLI options defaults,
but otherwise we already used the path to files from functions which
were already in the I/O monad, so we don't have to change much of the
code flow.

Additionally, Path.hs now has an explicit export list, to stop
accidental leakage of symbols.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

8 years agoChange keyToFilename/readSSconfFile interaction
Iustin Pop [Thu, 8 Nov 2012 17:48:19 +0000 (18:48 +0100)]
Change keyToFilename/readSSconfFile interaction

Currently, keyToFilename uses itself the default config path. In the
light of making that an function in the IO monad, let's remove the
"default" path functionality from this function and make it always
require the config path; its caller, readSSConfFile, can then do the
maybe/default path change.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoChange type of program options to 'IO [Options]'
Iustin Pop [Thu, 8 Nov 2012 14:04:10 +0000 (15:04 +0100)]
Change type of program options to 'IO [Options]'

Some options have defaults that depend on the environment, and we
could handle these in two ways:

- use a place-holder value (e.g. data X a = Default | Custom a) that
  is later read from the environment
- move the options list to IO monad, where it can read the
  environment, etc.

The second option allows also displaying the actual defaults in the
`--help' output, even though it's not as nice, so I went with it.

This patch only changes the option types, without actually changing
any options yet.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

8 years agoTwo small logging improvements
Iustin Pop [Tue, 6 Nov 2012 14:14:55 +0000 (15:14 +0100)]
Two small logging improvements

The "starting" message is changed to match the Python one, and in case
the preparation fails, we also log the error (beside printing it on
stderr or writing it to the error reporting pipe), as at this time
logging is usually set up.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoAnnotate confd startup/luxi binding error
Iustin Pop [Tue, 6 Nov 2012 14:06:52 +0000 (15:06 +0100)]
Annotate confd startup/luxi binding error

The luxi binding is now annotated for better readability.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoSwitch writing of PID file to new reporting style
Iustin Pop [Tue, 6 Nov 2012 13:53:45 +0000 (14:53 +0100)]
Switch writing of PID file to new reporting style

Currently, the writing of the PID file uses a "standard" error
reporting: catch exception, transform it into a 'Result' type, leave
handling of that to the caller. However, for daemon startup, we
actually want exceptions to be propagated up until the handler which
will write the details to the pipe.

This patch removes the writePidFile wrapper (and stops exporting it),
and changes the code to simply annotate better the I/O error.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoMove and rename BasicTypes.annotateIOError
Iustin Pop [Tue, 6 Nov 2012 13:49:53 +0000 (14:49 +0100)]
Move and rename BasicTypes.annotateIOError

It turns out that annotateIOError already exists in the standard
library, with a different purpose (this made me waste 10 minutes
trying to understand why the types were wrong…). Let's move this out
of BasicTypes, since it's used only in Daemon.hs, and also rename it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoSwitch daemon startup to pipe-based error reporting
Iustin Pop [Fri, 2 Nov 2012 16:01:30 +0000 (17:01 +0100)]
Switch daemon startup to pipe-based error reporting

This changes the daemon reporting error to the same mechanism as in
Python: use a pipe which is written to from the forked children.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoMove preparation steps of confd in prepMain
Iustin Pop [Fri, 2 Nov 2012 13:37:47 +0000 (14:37 +0100)]
Move preparation steps of confd in prepMain

This does the address parsing earlier, before we fork, for
better/faster error reporting.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoSplit queryd run into prepare and exec
Iustin Pop [Thu, 1 Nov 2012 14:38:53 +0000 (15:38 +0100)]
Split queryd run into prepare and exec

This will help with the general daemon split of prepare/run, and flag
errors earlier in the startup.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoConvert tag objects to a safer type
Iustin Pop [Thu, 8 Nov 2012 12:51:11 +0000 (13:51 +0100)]
Convert tag objects to a safer type

Currently, we keep information about the "target" of a tag operation
in a data type similar to (TagKind, Maybe String). This is unsafe, as
nothing (at the type level) prevents us from accidentally having
(TagCluster, Just "instance1.example.com"), or (TagInstance, Nothing).

To fix this problem, we rename the current TagObject type to TagType
(an internal utility type), and create TagObject as a better/safer
data type (see the definition), which doesn't allow such possibilities
in the future.

The downside is that, since at encoding level (both opcode and luxi)
this is done in an ugly way (type elements spread at the same level as
level as other value), we have to add custom encoders/decoders. The
encoder is shared between the OpCode and Luxi usage, the decoder is
different however as Luxi uses custom decoding.

This also fixes the recent breakage in confd w.r.t. QueryTags.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoSwitch Luxi TH code from simple to custom fields
Iustin Pop [Thu, 8 Nov 2012 12:26:40 +0000 (13:26 +0100)]
Switch Luxi TH code from simple to custom fields

This is needed so that we have more flexibility in generating Luxi
serialisation code (deserialisation is still custom). Also, only
exceptions are now using the 'simple' field types, so we might be able
later to convert and remove that TH code as well.

Since we will use custom serialisation fields in the future, we change
the order of serialisation for custom-save fields; Luxi uses
positional as opposed to name-based ordering, so we need to keep this
stable.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

8 years agoFix display of results in ==? operator
Iustin Pop [Thu, 8 Nov 2012 12:19:58 +0000 (13:19 +0100)]
Fix display of results in ==? operator

Doh, this is exactly the opposite of what we wanted… good thing no
test failed so far :)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoMake THH:genSaveOpCode a bit more general
Iustin Pop [Thu, 8 Nov 2012 12:08:32 +0000 (13:08 +0100)]
Make THH:genSaveOpCode a bit more general

This can be improved, by taking all hardcoded names as parameters, to
serve as a more-general "build save clause for a multi-constructor
data type". I'm not renaming the function as well, since I don't know
exactly how much we can abstract later.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

8 years agosphinx_ext: Allow use of “rapi” module in pyeval
Michael Hanselmann [Thu, 8 Nov 2012 18:08:07 +0000 (19:08 +0100)]
sphinx_ext: Allow use of “rapi” module in pyeval

This way constants like “rapi.RAPI_ACCESS_WRITE” can be used in
documentation.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agorlib2: Document two previously undocumented functions
Michael Hanselmann [Thu, 8 Nov 2012 17:51:03 +0000 (18:51 +0100)]
rlib2: Document two previously undocumented functions

Commit 208a6cff just included empty docstrings.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agojqueue/mcpu: Determine priority using callback
Michael Hanselmann [Wed, 24 Oct 2012 01:05:05 +0000 (03:05 +0200)]
jqueue/mcpu: Determine priority using callback

Instead of being given the priority for acquiring locks by means of a
parameter, mcpu will now call back. This is in preparation for
implementing a command to change a job's priority on the fly and allows
to change it while locks are being acquired (taking effect on the next
lock acquire).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

8 years agoMerge branch 'devel-2.6'
Michael Hanselmann [Thu, 8 Nov 2012 14:08:47 +0000 (15:08 +0100)]
Merge branch 'devel-2.6'

* devel-2.6:
  http/__init__.py: Remove extraneous argument
  rapi.testutils: Add utility to format HTTP headers
  rapi.testutils: Return headers from mock utility
  http: Add wrapper for mimetools.Message
  Add missing tests for commit f0d2286

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoMerge branch 'stable-2.6' into devel-2.6
Michael Hanselmann [Thu, 8 Nov 2012 13:56:35 +0000 (14:56 +0100)]
Merge branch 'stable-2.6' into devel-2.6

* stable-2.6:
  http/__init__.py: Remove extraneous argument
  rapi.testutils: Add utility to format HTTP headers
  rapi.testutils: Return headers from mock utility
  http: Add wrapper for mimetools.Message
  Add missing tests for commit f0d2286

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agohttp/__init__.py: Remove extraneous argument
Michael Hanselmann [Thu, 8 Nov 2012 13:45:46 +0000 (14:45 +0100)]
http/__init__.py: Remove extraneous argument

pylint complained, I fixed it, and unfortunately pushed too early.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agorapi.testutils: Add utility to format HTTP headers
Michael Hanselmann [Wed, 7 Nov 2012 17:34:36 +0000 (18:34 +0100)]
rapi.testutils: Add utility to format HTTP headers

Once again this will be used by forthcoming RAPI test.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agorapi.testutils: Return headers from mock utility
Michael Hanselmann [Wed, 7 Nov 2012 16:22:35 +0000 (17:22 +0100)]
rapi.testutils: Return headers from mock utility

A newly added test for RAPI will also verify the returned headers. A
test in ganeti.rapi.client_unittest.py is split into smaller stand-alone
tests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agohttp: Add wrapper for mimetools.Message
Michael Hanselmann [Wed, 7 Nov 2012 16:19:46 +0000 (17:19 +0100)]
http: Add wrapper for mimetools.Message

A newly added piece of code will also have to parse headers, so having
this wrapper saves us from copying this part of code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoRename Confd.hs to Confd/Types.hs
Iustin Pop [Thu, 8 Nov 2012 10:16:36 +0000 (11:16 +0100)]
Rename Confd.hs to Confd/Types.hs

This should be the last module rename, promise!

We rename this to conform to the other hierarchies (e.g. Query), and
to not have both Confd.hs and Confd/*.hs.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoOpCodes.hs: add OpTagsSet and OpTagsDel
Dato Simó [Wed, 7 Nov 2012 19:49:13 +0000 (19:49 +0000)]
OpCodes.hs: add OpTagsSet and OpTagsDel

Also, adjust comment to $(genOpCode) block to avoid repetition of
"only".

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agohtools: move TagObject from Luxi.hs to OpCodes.hs
Dato Simó [Wed, 7 Nov 2012 19:49:12 +0000 (19:49 +0000)]
htools: move TagObject from Luxi.hs to OpCodes.hs

In addition to ReqQueryTags in Luxi.hs, the TagObject ADT is also
required for the "kind" attribute of OpTagsSet and OpTagsDel, which
are coming to OpCodes.hs next. Hence, we move TagObject there, and
adjust imports accordingly.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd missing tests for commit f0d2286
Michael Hanselmann [Wed, 7 Nov 2012 15:40:47 +0000 (16:40 +0100)]
Add missing tests for commit f0d2286

Commit f0d2286 changed the logic of
gnt_instance._ConvertNicDiskModifications to also allow a parameter
named “modify”. Unfortunately the corresponding unittest was not
updated. An “if”/“else” condition is also merged.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoImplement base checkFn/prepFn/execFn model for daemons
Iustin Pop [Thu, 1 Nov 2012 11:47:01 +0000 (12:47 +0100)]
Implement base checkFn/prepFn/execFn model for daemons

This mirrors the code in the Python code base, and is required for
clean error reporting during startup.

This patch implements the basic infrastructure; the confd daemon is
not yet modified to take advantage of this, just the types are
adjusted.

Also, the pipe-based error reporting will go in a future patch, once
we can actually use that for reporting.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoOne more ghc 7.6 fix
Iustin Pop [Wed, 7 Nov 2012 13:30:42 +0000 (14:30 +0100)]
One more ghc 7.6 fix

This is only in master, so needed to be fixed separately.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoworkerpool: Use itertools.count instead of manual counting
Michael Hanselmann [Fri, 26 Oct 2012 15:38:03 +0000 (17:38 +0200)]
workerpool: Use itertools.count instead of manual counting

Instead of having to explicitely increment the value (“… += 1”), a call
to next() is enough. These numbers should in no case be re-used (they
are used for ordering tasks). Using “itertools.count” is useful here as
it guarantees that a returned number won't be returned another time.
Manual code for this could, over the course of time, gain unintended
bugs.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

8 years agoMerge branch 'devel-2.6' into submit
Iustin Pop [Wed, 7 Nov 2012 12:32:42 +0000 (13:32 +0100)]
Merge branch 'devel-2.6' into submit

* devel-2.6:
  Fix compatibility with newer Haskell libraries
  Fix gnt-instance console with xl

Conflicts:
        Makefile.am (reordering, fixed)
        htools/Ganeti/Confd/Server.hs (hlint fixes on master)
        htools/Ganeti/Daemon.hs (hlint)
        htools/Ganeti/HTools/Backend/Rapi.hs (hlint)
        htools/Ganeti/HTools/ExtLoader.hs (hlint)
        htools/Ganeti/HTools/QC.hs (file renamed/split in master, fixed)
        htools/test.hs (we don't use maxDiscards, ignored)
        lib/constants.py (move to pathutils)
        lib/hypervisor/hv_xen.py (move to pathutils)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoMerge branch 'stable-2.6' into devel-2.6
Iustin Pop [Wed, 7 Nov 2012 12:03:49 +0000 (13:03 +0100)]
Merge branch 'stable-2.6' into devel-2.6

* stable-2.6:
  Fix compatibility with newer Haskell libraries
  Fix gnt-instance console with xl

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoSwitch devel/upload to a static file
Iustin Pop [Wed, 7 Nov 2012 11:49:11 +0000 (12:49 +0100)]
Switch devel/upload to a static file

We had twice in the past days questions about devel/upload being
"broken", since bash re-reads shell scripts during their run and this
file can get regenerated due to Makefile changes.

Since we only need this to be dynamically built for 3 variables, let's
make the file static and read those three variables when it is run,
instead of when it is built, which allows us to re-read the "latest
version" of these vars as well.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoFix compatibility with newer Haskell libraries
Iustin Pop [Tue, 6 Nov 2012 21:25:51 +0000 (22:25 +0100)]
Fix compatibility with newer Haskell libraries

This small patch fixes compatibility with a few newer Haskell libraries:

- base 4.6, included with ghc 7.6, removed the deprecated 'catch'
  function from Prelude, so our "import Prelude hiding (catch)" is now
  an error; we workaround by using fully-qualified
  Control.Exception.catch name

- containers 0.5 changed the signature of 'deleteFindMax'; we
  workaround by using separate 'findMax' and 'deleteMax'

- QuickCheck 2.5 removed the 'maxDiscards' test parameter, replacing
  it with a much better 'maxDiscardsRatio'; however, until we can
  depend on that, we workaround by just removing it (we don't control
  anymore the maxDiscards, instead leaving it default; for our default
  test size, this is no change, as the default value is already 500,
  which is our default as well) and not printing it anymore

Tested on Squeeze (+extra libs), Wheezy and experimental, which covers
all supported GHC versions.

Also, merging this in master will be a pain, but unless we want to
stop supporting 2.6…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoUse SSH_LOGIN_USER rather than root for xl ssh
Guido Trotter [Tue, 6 Nov 2012 17:52:59 +0000 (18:52 +0100)]
Use SSH_LOGIN_USER rather than root for xl ssh

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoFix gnt-instance console with xl
Guido Trotter [Tue, 6 Nov 2012 16:54:04 +0000 (17:54 +0100)]
Fix gnt-instance console with xl

- Rename xm-console-wrapper to xen-console-wrapper
- Pass the xen command to use as a parameter

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd utility to check if file is executable
Michael Hanselmann [Tue, 6 Nov 2012 17:27:39 +0000 (18:27 +0100)]
Add utility to check if file is executable

This replaces direct calls to “os.access” and
“os.path.exists”/“os.path.isfile”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoFix NameError in constants.py introduced in merge 46c1f82
Michael Hanselmann [Tue, 6 Nov 2012 17:12:11 +0000 (18:12 +0100)]
Fix NameError in constants.py introduced in merge 46c1f82

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd test for Luxi calls consistency hs/py
Iustin Pop [Tue, 6 Nov 2012 15:47:53 +0000 (16:47 +0100)]
Add test for Luxi calls consistency hs/py

This tests that the same Luxi calls are defined in Python and
Haskell. It doesn't test yet that their serialisation is correct
though.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoMerge branch 'devel-2.6'
Guido Trotter [Tue, 6 Nov 2012 16:22:40 +0000 (17:22 +0100)]
Merge branch 'devel-2.6'

* devel-2.6:
  Disable E1101 on ganeti/http/server.py:424
  Fix live migration under xl
  Don't check for xend port when using xl

Conflicts:
lib/hypervisor/hv_xen.py: trivial

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoMerge branch 'stable-2.6' into devel-2.6
Guido Trotter [Tue, 6 Nov 2012 16:09:40 +0000 (17:09 +0100)]
Merge branch 'stable-2.6' into devel-2.6

* stable-2.6:
  Disable E1101 on ganeti/http/server.py:424
  Fix live migration under xl
  Don't check for xend port when using xl

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoDisable E1101 on ganeti/http/server.py:424
Guido Trotter [Tue, 6 Nov 2012 15:35:34 +0000 (16:35 +0100)]
Disable E1101 on ganeti/http/server.py:424

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoFix live migration under xl
Guido Trotter [Mon, 5 Nov 2012 14:18:33 +0000 (15:18 +0100)]
Fix live migration under xl

Until now the only way to make live migration work in conjunction with
"xl" was to add ssh known_hosts keys for every node's secondary ip on
every other node.

With this command we remove the target key verification: this is not
worse than what we were doing before with "xm", and allows the migration
to happen under either toolstack, without extra manual work. Of course
the full security of ssh is not used by live migration, then.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoDon't check for xend port when using xl
Guido Trotter [Mon, 5 Nov 2012 13:32:24 +0000 (14:32 +0100)]
Don't check for xend port when using xl

If the toolstack is set to "xl" we shouldn't ping xend for liveness
before attempting a live migration.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoutils.io: Improve handling of double and single slashes
Michael Hanselmann [Mon, 5 Nov 2012 18:12:51 +0000 (19:12 +0100)]
utils.io: Improve handling of double and single slashes

Up until now “IsBelowDir("/", …)” would never return True. The reason
was that an additional slash was added to the root path resulting in
“//", which is “implementation-defined” in posix and treated specially
by “os.path.normpath”.

This patch fixes the behaviour for this special case and adds tests
(also for IsNormAbsPath). A typo in the docstring is fixed. Calls to
“assert_” and “assertFalse” are changed to pass a message by keyword
argument.

It is a bit of a mess, but I hope the resulting behaviour is correct.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Dato Simó <dato@google.com>

8 years agoworkerpool: Don't mask variable in AddManyTasks
Michael Hanselmann [Fri, 2 Nov 2012 17:07:28 +0000 (18:07 +0100)]
workerpool: Don't mask variable in AddManyTasks

The name “priority” is already used.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

8 years agoworkerpool: Simplify _WaitForTaskUnlocked
Michael Hanselmann [Tue, 30 Oct 2012 15:43:24 +0000 (16:43 +0100)]
workerpool: Simplify _WaitForTaskUnlocked

The function in is simplified in its structure and duplicated checks
have been merged.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

8 years agocli.py: use None as name for tag operations on the cluster
Dato Simó [Mon, 5 Nov 2012 17:54:59 +0000 (17:54 +0000)]
cli.py: use None as name for tag operations on the cluster

This change is mostly cosmetic. Previously, the literal "cluster" was
used for the 'name' field of tag operations on the cluster (as opposed
to a node or an instance). Since this field has a type of TMaybeString
specifically for the case of the cluster, it seems more correct to use
None, rather than an arbitrary string (that is not used by the callee).

Additionally: note in opcodes.py that groups also expect a name; the
previous comment only referred to nodes and instances.

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoFix previous merge
Bernardo Dal Seno [Thu, 1 Nov 2012 20:45:18 +0000 (21:45 +0100)]
Fix previous merge

A call to _CalculateGroupIPolicy wasn't refactored during the merge.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoMerge branch 'devel-2.6'
Michael Hanselmann [Thu, 1 Nov 2012 17:27:27 +0000 (18:27 +0100)]
Merge branch 'devel-2.6'

* devel-2.6:
  Fix typo in gnt-instance man page
  jqueue: Return jobs to queue when shutting down
  gnt-debug delay: Add "--submit" option
  Make hostname checks uniform between instance rename and add
  Improve logging of new job submissions
  Improve handling of lock exceptions
  Add note about developing on a production machine
  Fix runtime memory increases
  Fix validation of vgname in OpClusterSetParams
  Fix removal of storage directory on shared file storage
  Switch non-redundant check to disk template-based
  Document the new --yes-do-it option for master-failover
  Add option to force master-failover without voting
  Update instance modify message
  Force tap's MAC prefix to "fe"
  Fix disk adoption interaction with ipolicy checks
  Better NEWS file for 2.6.1

Conflicts:
lib/cmdlib.py: Trivial
tools/kvm-ifup.in: Space vs. tab

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoMerge branch 'stable-2.6' into devel-2.6
Michael Hanselmann [Thu, 1 Nov 2012 15:59:01 +0000 (16:59 +0100)]
Merge branch 'stable-2.6' into devel-2.6

* stable-2.6:
  Fix typo in gnt-instance man page
  jqueue: Return jobs to queue when shutting down
  gnt-debug delay: Add "--submit" option
  Make hostname checks uniform between instance rename and add
  Improve logging of new job submissions
  Improve handling of lock exceptions
  Add note about developing on a production machine
  Fix runtime memory increases
  Fix validation of vgname in OpClusterSetParams
  Fix removal of storage directory on shared file storage
  Switch non-redundant check to disk template-based
  Document the new --yes-do-it option for master-failover
  Add option to force master-failover without voting
  Update instance modify message
  Force tap's MAC prefix to "fe"
  Fix disk adoption interaction with ipolicy checks
  Better NEWS file for 2.6.1

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoFix typo in gnt-instance man page
Michael Hanselmann [Thu, 1 Nov 2012 15:04:33 +0000 (16:04 +0100)]
Fix typo in gnt-instance man page

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agojqueue: Return jobs to queue when shutting down
Michael Hanselmann [Thu, 1 Nov 2012 13:54:03 +0000 (14:54 +0100)]
jqueue: Return jobs to queue when shutting down

When a job is still waiting for locks and the queue is shutting down,
they should be returned and not actually start processing. Until now
jobs which transitioned from “queued” to “waiting” were already
considered to be running as far as the shutdown code was concerned.

This fixes issue 296.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agognt-debug delay: Add "--submit" option
Michael Hanselmann [Thu, 1 Nov 2012 14:23:54 +0000 (15:23 +0100)]
gnt-debug delay: Add "--submit" option

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoRemove duplicate workerpool test
Michael Hanselmann [Wed, 31 Oct 2012 13:17:09 +0000 (14:17 +0100)]
Remove duplicate workerpool test

Commit 52c47e4e (July 2010) added the exact test twice, probably due to
a copy & paste error.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoMake hostname checks uniform between instance rename and add
Iustin Pop [Wed, 31 Oct 2012 16:10:19 +0000 (17:10 +0100)]
Make hostname checks uniform between instance rename and add

Currently, we have instance rename doing extra checks on the host
name, to prevent accidental wrong renames; however, instance create
doesn't do these checks (issue 291), which (if DNS is misconfigured)
can lead to hard to diagnose errors.

This patch abstracts the name checking from LUInstanceRename into a
separate function, which is then reused in both instance rename and
instance create.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

8 years agoImprove logging of new job submissions
Iustin Pop [Wed, 31 Oct 2012 13:47:18 +0000 (14:47 +0100)]
Improve logging of new job submissions

This addresses issue 290: when receiving new jobs, logging is
incomplete, and we don't have the job ID(s) and/or summaries
logged. Only later, when the job is queried for or being processed, we
know more.

This is not good when troubleshooting, so let's improve the initial
logging.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

8 years agoImprove handling of lock exceptions
Iustin Pop [Wed, 31 Oct 2012 13:31:48 +0000 (14:31 +0100)]
Improve handling of lock exceptions

There are two issues with lock exceptions right now:

- first, we don't log the original error; this is fine for now
  (locking.py always returns the same error here), but in general is
  brittle: if locking.py would start returning more information, we'd
  completely miss that

- second, an actual honest lock conflict is not an internal error;
  it's simply an optimistic lock failing, and as such we should not
  return internal error, but rather resource_not_unique

This addresses issue 287.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

8 years agoAdd note about developing on a production machine
Iustin Pop [Tue, 30 Oct 2012 15:49:33 +0000 (16:49 +0100)]
Add note about developing on a production machine

This is the bit of documentation missing for issue 170. Doing
development on a machine which already has Ganeti installed kind of
works, but only when the installed and the developed version are very
similar, and even then it can be problematic.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoFix runtime memory increases
Iustin Pop [Tue, 30 Oct 2012 13:43:09 +0000 (14:43 +0100)]
Fix runtime memory increases

Commit 2c0af7da which added the runtime memory changes functionality
had a small typo (wrong name); I've rewritten this to only compute the
delta once, for simplicity.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoFix validation of vgname in OpClusterSetParams
Iustin Pop [Tue, 30 Oct 2012 11:53:26 +0000 (12:53 +0100)]
Fix validation of vgname in OpClusterSetParams

This variable can be empty, when we want to disable LVM, so we can't
use TMaybeString.

Fixes issue 285.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoFix removal of storage directory on shared file storage
Iustin Pop [Tue, 30 Oct 2012 11:29:16 +0000 (12:29 +0100)]
Fix removal of storage directory on shared file storage

This patch makes _RemoveDisks symmetric to _CreateDisks with respect
to file-based storage: _CreateDisks uses "in constants.DTS_FILEBASED",
whereas _RemoveDisks was not update and only uses "==
constants.DT_FILE". This results in stale directories left on the
filesystem.

Fixes issue 262.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoSwitch non-redundant check to disk template-based
Iustin Pop [Tue, 30 Oct 2012 11:14:37 +0000 (12:14 +0100)]
Switch non-redundant check to disk template-based

Currently, the warning/notice about non-redundant instances in cluster
verify is based non empty secondaries list (how old is this?); the
proper way to check this nowadays is via DTS_MIRRORED.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoFix permission for socket directory
Bernardo Dal Seno [Mon, 29 Oct 2012 17:44:35 +0000 (18:44 +0100)]
Fix permission for socket directory

The directory must we writable also by the confd daemon user.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoDocument the new --yes-do-it option for master-failover
Iustin Pop [Mon, 29 Oct 2012 12:01:58 +0000 (13:01 +0100)]
Document the new --yes-do-it option for master-failover

Sorry, this should have gone in the previous commit, I forgot about
it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

8 years agoAdd option to force master-failover without voting
Iustin Pop [Mon, 29 Oct 2012 11:21:59 +0000 (12:21 +0100)]
Add option to force master-failover without voting

This fixes issue 282.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

8 years agoConvert Luxi results to Ganeti errors
Iustin Pop [Fri, 26 Oct 2012 13:27:11 +0000 (15:27 +0200)]
Convert Luxi results to Ganeti errors

This a bit too complex patch converts the result of Luxi calls
(submitJob, query*, etc.) from Result to ErrorResult. It then
immediately revers this in the HTools/Backend/Luxi module, where we
don't need necessarily the full error type (just a nice error
message), and does the same in Hbal's job execution functions.

While at first sight this doesn't seem to do much, what we get is
actual error messages from Ganeti, plus improvements to the result
parsing: instead of "can't parse char", we now get properly (note,
wrapped manually):

  Executing jobset for instances instance1, …
  Job submission error: Failure: the job queue is marked for drain and
    doesn't accept new requests

Or:

  Job submission error: Unhandled exception: LuxiError "parsing job
    id: cannot parse string 'a956101'"

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agocfgupgrade: Write file for file storage paths
Michael Hanselmann [Fri, 5 Oct 2012 01:32:44 +0000 (03:32 +0200)]
cfgupgrade: Write file for file storage paths

When file storage is used this file is now mandatory.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agobackend: Switch to new file storage directory verification
Michael Hanselmann [Fri, 5 Oct 2012 00:59:12 +0000 (02:59 +0200)]
backend: Switch to new file storage directory verification

The configuration is no longer used for verifying file storage paths.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoCheck allowed file storage paths during cluster-verify
Michael Hanselmann [Fri, 5 Oct 2012 00:43:47 +0000 (02:43 +0200)]
Check allowed file storage paths during cluster-verify

Some paths, such as /bin or /usr/lib, should not be used for file
storage. This patch implements a check during cluster verification to
fail in case such a path has been used.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoMake Paramiko an optional dependency for listrunner
Michael Hanselmann [Wed, 24 Oct 2012 00:01:42 +0000 (02:01 +0200)]
Make Paramiko an optional dependency for listrunner

With the move away from “setup-ssh”, Paramiko is no longer necessary to
configure SSH on nodes.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoRemove setup-ssh
Michael Hanselmann [Tue, 23 Oct 2012 23:58:37 +0000 (01:58 +0200)]
Remove setup-ssh

It has been superseeded by “prepare-node-join”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agognt-node add: Use prepare-node-join
Michael Hanselmann [Tue, 23 Oct 2012 18:16:25 +0000 (20:16 +0200)]
gnt-node add: Use prepare-node-join

This patch changes “gnt-node add” to use the newly added
“prepare-node-join” tool. Hereby Paramiko is no longer a hard dependency
for setting up SSH on nodes.

In “gnt_cluster.py”, a positional parameter is no longer passed as a
keyword parameter.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoprepare-node-join: Use ssh.GetAllUserFiles
Michael Hanselmann [Tue, 23 Oct 2012 23:16:50 +0000 (01:16 +0200)]
prepare-node-join: Use ssh.GetAllUserFiles

Instead of building the dictionary locally, the global version in
“ssh.py” can be used.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agossh: Add function to get all of user's SSH files
Michael Hanselmann [Tue, 23 Oct 2012 23:10:36 +0000 (01:10 +0200)]
ssh: Add function to get all of user's SSH files

This new function returns the file paths for all of a user's SSH-related
files (RSA, DSA and authorized_keys).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoRunCmd: Support standard input file descriptor
Michael Hanselmann [Tue, 23 Oct 2012 19:25:30 +0000 (21:25 +0200)]
RunCmd: Support standard input file descriptor

This patch changes “utils.RunCmd” to accept a file-like object or a
numeric file descriptor which will be used as the command's standard
input. One use-case will be to pass all necessary data to
“prepare-node-join”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoFactorize job selection in “gnt-job cancel”
Michael Hanselmann [Thu, 25 Oct 2012 15:29:51 +0000 (17:29 +0200)]
Factorize job selection in “gnt-job cancel”

This will also be used for changing jobs' priorities. All parameters to
the common function are non-optional.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoutils.x509: Factorize code to extract X509 certificate
Michael Hanselmann [Tue, 23 Oct 2012 23:55:53 +0000 (01:55 +0200)]
utils.x509: Factorize code to extract X509 certificate

This will be useful in “gnt-node add”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoprepare_node_join: Move daemon SSH files to constants
Michael Hanselmann [Tue, 23 Oct 2012 23:11:45 +0000 (01:11 +0200)]
prepare_node_join: Move daemon SSH files to constants

This dictionary will also be useful in “gnt-node add”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoprepare-node-join: Swap private and public keys
Michael Hanselmann [Tue, 23 Oct 2012 23:24:19 +0000 (01:24 +0200)]
prepare-node-join: Swap private and public keys

Other places, such as “ssh.GetUserFiles”, use a structure where the
private key comes before the private key. Until now prepare-node-join
did the opposite, that is the public key came first. To avoid confusion
and potential bugs, this is changed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoprepare-node-join: Use public key directly for auth…_keys
Michael Hanselmann [Tue, 23 Oct 2012 22:26:16 +0000 (00:26 +0200)]
prepare-node-join: Use public key directly for auth…_keys

A public key already includes the necessary prefix (“ssh-rsa” or
“ssh-dss”), so there is no need to add it again.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agossh.GetUserFiles: Parameter to disable directory check
Michael Hanselmann [Tue, 23 Oct 2012 22:55:39 +0000 (00:55 +0200)]
ssh.GetUserFiles: Parameter to disable directory check

Without this parameter, either an error would be raised or “.ssh” would
have to be created. Now it is possible to retrieve the paths without
requiring the “.ssh” directory to exist.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoMove htools backends to a separate directory
Iustin Pop [Fri, 26 Oct 2012 07:32:35 +0000 (09:32 +0200)]
Move htools backends to a separate directory

Five modules under the HTools/ directories are backend
implementations, so let's move them to a separate directory, to more
clearly show the hierarchy. I wanted to do this for a while, but
merging between branches is always an issue, so let's do it know since
we have an opportunity.

This patch contains the actual renames, the required changed module
names, imports, etc., but no other changes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoFix lint issue in Test/…/THH.hs
Iustin Pop [Fri, 26 Oct 2012 11:48:45 +0000 (13:48 +0200)]
Fix lint issue in Test/…/THH.hs

Sorry!

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoFix X509CertError definition in Haskell codebase
Iustin Pop [Fri, 26 Oct 2012 08:22:21 +0000 (10:22 +0200)]
Fix X509CertError definition in Haskell codebase

Thanks Dato for catching this.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoFix a few issues found by newer hlint
Iustin Pop [Fri, 26 Oct 2012 06:49:48 +0000 (08:49 +0200)]
Fix a few issues found by newer hlint

Testing with a newer hlint found a few minor issues; but all are real,
valid recommendations:

- don't use "if cond then f x else f y", but "f (if cond then x else y)"
- "if a then b else True" is equivalent to the simpler "not a || b"
- and as usual, one more ignore to our "testing basic properties"
  module

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoAdd a simple unittest for THH code
Iustin Pop [Thu, 25 Oct 2012 14:04:56 +0000 (16:04 +0200)]
Add a simple unittest for THH code

This is very THH specific, and applies to all serialisations generated
by THH, so I'm adding it in its own module.

Probably we should add some more generic tests, but in general THH
code is tested by the various definitions; this new field type however
is not (yet), so this is why I want this specific unittest.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoAdd support for optional fields with null serialised
Iustin Pop [Thu, 25 Oct 2012 13:48:05 +0000 (15:48 +0200)]
Add support for optional fields with null serialised

This follows a conversation we had for how to deal with
optional-but-required fields in JSON serialisations: fields which are
optional (can be either a given type or 'null'), but where the 'null'
value is required. There are just a few of these in the Python code,
but we should support them nevertheless.

The patch changes the 'isOptional' attribute from boolean to a custom
ADT, three-typed. This allows us to keep the same path on load (which
deals with both cases), but use a custom save path where we explicitly
save the 'null' value.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoUpdate instance modify message
Iustin Pop [Fri, 26 Oct 2012 06:41:07 +0000 (08:41 +0200)]
Update instance modify message

Currently the message does not say explicitly that instance-initiated
reboots are useless to trigger the use of new parameters, per the
thread on the user mailing list. Let's improve it a bit.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

8 years agoErrors.hs: improve field names for ConfigVersionMismatch
Dato Simó [Thu, 25 Oct 2012 16:11:12 +0000 (17:11 +0100)]
Errors.hs: improve field names for ConfigVersionMismatch

Change {exp,act}Code to {exp,act}Ver, which gives a better idea that
the integer fields represent version numbers.

Also:

  - errors.py: update OpPrereqError's docstring to note that an error
    code is always expected as the second argument (it was previously
    optional).

Signed-off-by: Dato Simó <dato@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>