ganeti-github.git
8 years agoAdd tags in network objects
Dimitris Aragiorgis [Fri, 27 Jul 2012 07:50:04 +0000 (10:50 +0300)]
Add tags in network objects

Support:
 - gnt-network add --tags.
 - gnt-network list-tags/add-tags/remove-tags/.
 - gnt-network list -o +tags

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoChange default fields for network listing
Dimitris Aragiorgis [Mon, 18 Jun 2012 16:05:28 +0000 (19:05 +0300)]
Change default fields for network listing

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoPass detailed network info in hooks
Dimitris Aragiorgis [Thu, 7 Jun 2012 15:51:34 +0000 (18:51 +0300)]
Pass detailed network info in hooks

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoPass netinfo in rpcs
Dimitris Aragiorgis [Wed, 6 Jun 2012 12:57:37 +0000 (15:57 +0300)]
Pass netinfo in rpcs

If a nic has a network field then encapsulate a network object in
netinfo slot for every rpc. This is needed to pass network info to
scripts managing nics (kvm-vif-bridge).

Introduce _BuildNetworkEnv().

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agocfgupgrade for networks
Dimitris Aragiorgis [Tue, 5 Jun 2012 23:08:11 +0000 (02:08 +0300)]
cfgupgrade for networks

Add networks slot in config data.
Add networks slot in nodegroups.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoUse network's mac prefix
Dimitris Aragiorgis [Tue, 5 Jun 2012 23:03:08 +0000 (02:03 +0300)]
Use network's mac prefix

Modify existing GenerateMAC so that it takes network as an argument.

Substitute _GenerateOneMAC with a decorator _GenerateMACPrefix that
chooses the prefix (network's or cluster's) and wraps
_GenerateMACSuffix that generates the three remaining bytes.

Check mac prefix validity when adding/modifing a network.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoInclude networks in Makefile and bash completion
Apollon Oikonomopoulos [Mon, 4 Jun 2012 20:52:31 +0000 (23:52 +0300)]
Include networks in Makefile and bash completion

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd gnt-network man page
Apollon Oikonomopoulos [Mon, 4 Jun 2012 20:51:44 +0000 (23:51 +0300)]
Add gnt-network man page

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoRapi support for networks
Dimitris Aragiorgis [Mon, 4 Jun 2012 20:20:09 +0000 (23:20 +0300)]
Rapi support for networks

Support:
 - GetNetwork(s)
 - CreateNetwork
 - ConnectNetwork
 - DisconnectNetwork
 - RemoveNetwork

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoss_conf support for networks
Apollon Oikonomopoulos [Mon, 4 Jun 2012 20:15:42 +0000 (23:15 +0300)]
ss_conf support for networks

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoIntroduce client support for networks
Dimitris Aragiorgis [Mon, 4 Jun 2012 20:09:27 +0000 (23:09 +0300)]
Introduce client support for networks

gnt-network is used to manipulate and handle networks that currently
provides the following operations:

 * Add a new network:
   gnt-network add --network=1.2.3.0/28 --gateway=1.2.3.1
                   --add-reserved-ips=1.2.3.4,1.2.3.5 testnet

 * Remove an existing network:
   gnt-network remove testnet

 * Modify an existing network:
   gnt-network modify --gateway=1.2.3.6
                      --network-type=private
                      --network6=2001:648::/64
                      --gateway6=none testnet
                      --add-reserved-ips=1.2.3.10,1.2.3.10,
                      --remove-reserved-ips=1.2.3.20
                      testnet

 * Connect an existing network to a nodegroup:
   gnt-network connect testnet default bridged br100
   gnt-network connect testnet <nodegroup> <mode> <link>
   (pass all for <nodegroup> to connect to all nodegroups)

 * Disconnect an existing network from a nodegroup:
   gnt-network disconnect testnet <nodegroup>
   (pass all for <nodegroup> to disconnect from all nodegroups)

 * List available networks:
   gnt-network list

 * Show network info:
   gnt-network info [testnet]

Introduce new option NOCONFLICTSCHECK_OPT for not checking for
conflicting IPs. Using this might cause data inconsistency.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoModify instance client to support networks
Apollon Oikonomopoulos [Mon, 4 Jun 2012 20:03:44 +0000 (23:03 +0300)]
Modify instance client to support networks

Add --no-conflicts-check option.
Modify instance query methods to return nic's network info.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoPeripheral changes for networks
Apollon Oikonomopoulos [Mon, 4 Jun 2012 19:58:12 +0000 (22:58 +0300)]
Peripheral changes for networks

Specifically:
 * export a new environment variable for the gnt-os-interface
 * export a new environment variable for KVM specific scripts
 * ovf support for networks

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoModify LUInstanceSetParams() to support networks
Dimitris Aragiorgis [Mon, 4 Jun 2012 19:39:53 +0000 (22:39 +0300)]
Modify LUInstanceSetParams() to support networks

Keep backwards compatibility just like LUInstanceCreate() and support
network parameter for NIC object.

Add ec_id argument in Update() to be able to commit temporary IPs that
were previously reserved.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoModify RemoveInstance() to support networks
Dimitris Aragiorgis [Mon, 4 Jun 2012 19:37:12 +0000 (22:37 +0300)]
Modify RemoveInstance() to support networks

Release any IPs held the instance before actually removing.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoModify LUInstanceCreate to support networks
Dimitris Aragiorgis [Mon, 4 Jun 2012 19:31:30 +0000 (22:31 +0300)]
Modify LUInstanceCreate to support networks

Implement backend support, to export the IP pool management
functionality to the clients. When the new NIC parameter 'network' is
given, the ippool management system is triggered.  If a NIC belongs to
a network, it inherits the netparams (mode, link) as its nicparams. If
a network is requested, then no 'mode' or 'link' should be
specified. Backwards compatibility is preserved, if 'network'
parameter is omitted.

Examples for IP pool management system:

--net 0:network=net1
  for an IPless NIC in a network

--net 0:ip=pool,network=net1
  for automatically generated IP inside the network

--net 0:ip=1.2.3.4,network=net1
  for specific IP inside the network

--net 0:network=none
  for IPless NIC with default nicparams

Examples for traditional assignment:

--net 0:ip=1.2.3.4,link=br100,mode=bridged
--net 0:ip=none,link=rt2,mode=routed
--net 0

Check for conflicting IPs to ensure IP uniqueness inside nodegroups.

IAllocator should be extended to become network aware. If an instance
has a NIC inside a network, then IAllocator must pick a node that its
nodegroup is already connected with that network.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoConfig methods for reserving/releasing IPs
Dimitris Aragiorgis [Mon, 4 Jun 2012 19:24:47 +0000 (22:24 +0300)]
Config methods for reserving/releasing IPs

Use TemporaryReservationManager for IP handling to ensure config data
consistency.

Implement:
 - methods for reserving/releasing IPs
 - _UnlockedCommitTemporaryIps()
 - CheckIPInNodeGroup() used for conflicting IPs
 - GetGroupNetParams() used for inheriting netparams.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoImplement network/nodegroup mappings
Dimitris Aragiorgis [Mon, 4 Jun 2012 18:08:36 +0000 (21:08 +0300)]
Implement network/nodegroup mappings

LUNetworkConnect() connects a network to a nodegroup and defines
its netparams (mode and link). Specifically:
 * Check if network already exists
 * Check if netparams are valid
 * Check if already connected to the same nodegroup
 * Produce errors if any conflicting IPs found
 * Update the 'networks' slot in NodeGroup config object
   with the network UUID and netparams requested. E.g.
   nodegroup_uuid {
     networks {
       network_uuid: {mode: 'bridged', link: 'br100'}
     }
   }

LUNetworkDisconnect() unmaps a network from a nodegroup:
 * Produce errors if an instance has a NIC with IP inside the network
 * Remove corresponding entry in 'networks' slot of NodeGroup object

Check for conflicting ips can be avoided with corresponding option.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoImplement LUNetworkSetParams
Dimitris Aragiorgis [Mon, 4 Jun 2012 18:05:03 +0000 (21:05 +0300)]
Implement LUNetworkSetParams

Support modifying all network parameters except for network (ip
range). Cannot modify gateway and reserved ips at the same time.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoImplement LUNetworkQuery
Apollon Oikonomopoulos [Mon, 4 Jun 2012 18:02:55 +0000 (21:02 +0300)]
Implement LUNetworkQuery

Summarily list all existing networks
Supply detailed info for every existing network
 - List used/free IPs
 - List instances with NICs assigned to the corresponding network
 - List NIC index and IP for the above instances

Implement complementary config methods for retrieving networks.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoBasic IP pool management logic
Dimitris Aragiorgis [Mon, 4 Jun 2012 15:54:02 +0000 (18:54 +0300)]
Basic IP pool management logic

Implement LUs for corresponding opcodes:
 * LUNetworkAdd:
   - Check for IP validity
   - Reserves all necessary IPs
   - Create new Network config object
 * LUNetworkRemove:
   - Checks if connected to any nodegroup
   - Remove a Network config object

Implement basic config methods:
 * LookupNetwork()
   - Given the network name return the network UUID
 * AddNetwork()
   - Add a new network to the config
 * RemoveNetwork()
   - Remove a network from the config

Add new locking level: LEVEL_NETWORK

Add various useful config methods for retrieving network info.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoIntroduce new module for IP pool management
Apollon Oikonomopoulos [Mon, 4 Jun 2012 14:33:22 +0000 (17:33 +0300)]
Introduce new module for IP pool management

Add new library module lib/network.py.
Introduce new class: AddressPool.

AddressPool implements all operations needed for managing IPs inside
the IP pool.

Given a Network config object (nobj), the class:

 * initializes the corresponding IP pool object via
   network.AddressPool.InitializeNetwork(nobj)
 * obtains the corresponding IP pool object via
   network.AddressPool(nobj)
 * manipulates IPs inside the pool

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoIP pool related objects, opcodes and constants
Dimitris Aragiorgis [Mon, 4 Jun 2012 13:50:42 +0000 (16:50 +0300)]
IP pool related objects, opcodes and constants

Config objects:
 * Introduce L{Network} with
  - IPv4 network field (mandatory)
  - IPv4 gateway, IPv6 (network/gateway), mac prefix, type (optional)
 * Modify existing config objects to support networks:
  - Add new slot 'network' to L{NIC} config object
  - Add new slot 'networks' to L{NodeGroup} config object

Opcodes:
 * Introduce new opcodes for networks
  - add/remove/modify/query/connect/disconnect.
 * In InstanceCreate/InstanceSetParams add conflicts_check option

Constants:
 * INIC_PARAM 'INIC_NETWORK'
 * NIC_IP_POOL for automaticaly obtain an IP from a pool
 * NETWORK_TYPE_PUBLIC/PRIVATE for network types

Checking of network_type handled by the opcode parameter validation.
Introduce _CheckCIDR*Notation() functions for network parameters
validation.

Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd more basic validation types
Iustin Pop [Thu, 15 Nov 2012 12:24:14 +0000 (13:24 +0100)]
Add more basic validation types

This mirrors the ht.py types PositiveInt, NonNegative, etc., except
that they work at a more generic level (any numeric type, respectively
any non-empty list).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

8 years agoFix a hlint warning
Iustin Pop [Tue, 20 Nov 2012 09:59:45 +0000 (10:59 +0100)]
Fix a hlint warning

This is yet-another version-specific warning (fixed in later hlint
versions), but it's better to keep it clean.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoAdd DRBD parser unit tests
Michele Tartara [Mon, 19 Nov 2012 14:20:36 +0000 (15:20 +0100)]
Add DRBD parser unit tests

This adds tests that existing test files can be parsed by the Haskell
parser as well, plus one new test file.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd parser for DRBD /proc file
Michele Tartara [Mon, 19 Nov 2012 14:19:51 +0000 (15:19 +0100)]
Add parser for DRBD /proc file

A new directory for haskell modules about block devices has been created
The parser is divided in two modules:
* one exports the data types describing the DRBD status
* one exports the parser itself

Signed-off-by: Michele Tartara <mtartara@google.com>
[iustin@google.com: indentation/alignment fixes]
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoProperly log errors when setting up daemon FDs
Iustin Pop [Tue, 6 Nov 2012 14:23:34 +0000 (15:23 +0100)]
Properly log errors when setting up daemon FDs

While writing the pipe-based reporting and trying various ways to
break the startup, I fought for a while trying to understand why error
reporting was _different_ when running the daemon as a user (with no
rights). It turns out that setupDaemonFDs wants to open the log file
in append mode way before, so we are not protected by the 'prepare'
phase.

This patch explicitly runs the 'setupDaemonFDs' function under the
same handler as the prepare phase, with the only change that here we
instruct handlePrepErr to not log the message via log*, since logging
is not yet set up.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoSwitch opcode data type from normal to record constructors
Iustin Pop [Mon, 19 Nov 2012 09:21:28 +0000 (10:21 +0100)]
Switch opcode data type from normal to record constructors

Currently, the OpCode definitions are using normal constructors:

  data OpCode = OpTestDelay Double Bool [String]
              | OpInstanceFailover String Bool (Maybe String)
              …

While this works for a few opcodes, it becomes unwieldy when dealing
with a bigger number of opcode definitions and/or with opcodes having
many fields.

This patch changes the opcodes to record-based constructors, so that
we get for free accessor functions:

  data OpCode
    = OpTestDelay {
        opDuration :: Double,
        opOnMaster :: Bool,
        opOnNodes :: [String]
      }
      | OpInstanceFailover {
         opInstanceName :: String,
         opIgnoreConsistency :: Bool,
         opTargetNode :: Maybe String
      }
      …

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

8 years agoFix docstrings for the Filter type
Iustin Pop [Mon, 19 Nov 2012 16:07:01 +0000 (17:07 +0100)]
Fix docstrings for the Filter type

While looking at the opcode docs and clicking accidentally on the
filter type, I saw that the haddock formatting is broken due to
non-escaped use of special chars. Let's convert the ascii-like
formatting to haddock, and have nicer apidoc.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoSimplify a few test helpers
Iustin Pop [Mon, 19 Nov 2012 15:57:24 +0000 (16:57 +0100)]
Simplify a few test helpers

Use more monadic combinators instead of explicit code.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

8 years agoAdd custom code for CV_E* constants to convert-constants
Iustin Pop [Fri, 16 Nov 2012 13:24:20 +0000 (14:24 +0100)]
Add custom code for CV_E* constants to convert-constants

Currently, the cluster verify errors are defined as follows:

CV_ECLUSTER_FOO = (TCLUSTER, "ECLUSTER_FOO", "description")

This means there's no standalone name for the string "ECLUSTER_FOO",
which makes it hard to derive automatically a type for this union in
Haskell.

There are three possible fixes:

- manually separate the CV_ECLUSTER_FOO_STR = "ECLUSTER_FOO" in
  constants.py
- manually extract the strings in Haskell code
- change convert-constants to automatically export virtual constants
  for those

After discussion on IRC, I've taken the latter approach; even though a
bit hack-ish, it avoids manual work and potential errors.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoImprove error message for wrong opcode parameter values
Iustin Pop [Mon, 19 Nov 2012 09:10:09 +0000 (10:10 +0100)]
Improve error message for wrong opcode parameter values

When given an empty string as value to a parameter that doesn't accept
empty strings, the error message was confusing, since it was showing
just as a double space.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

8 years agoCleanup ht's use of positive/strictpositive
Iustin Pop [Thu, 15 Nov 2012 09:28:17 +0000 (10:28 +0100)]
Cleanup ht's use of positive/strictpositive

Currently, ht.py uses a bad terminology for positive/non-negative
numbers. Per http://en.wikipedia.org/wiki/Positive_number, this is the
correct terminology:

- A number is positive if it is greater than zero.
- A number is negative if it is less than zero.
- A number is non-negative if it is greater than or equal to zero.
- A number is non-positive if it is less than or equal to zero.

So this patch renames things as follows:

- TPositiveInt            ⇒ TNonNegativeInt
- TStrictPositiveInt      ⇒ TPositiveInt
- TMaybePositiveInt       ⇒ dropped, not used anywhere
- TMaybeStrictPositiveInt ⇒ TMaybePositiveInt
- TPositiveFloat          ⇒ TNonNegativeFloat
- TStrictNegativeInt      ⇒ TNegativeInt

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoSplit OpCode.hs and add module for opcode parameters
Iustin Pop [Tue, 13 Nov 2012 15:37:31 +0000 (16:37 +0100)]
Split OpCode.hs and add module for opcode parameters

Due to TemplateHaskell stage restrictions, we can't define parameters
in the same module as we're using them for TH, so we have to define
all module parameters in a separate module.

This patch therefore splits OpCodes.hs in two, adding that module and
moves most code there (types, parameters, etc.). The remaining parts
in OpCodes.hs, the actual opcode definitions, now use more parameters
instead of direct field definitions (more will come later)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

8 years agoCreate a new Ganeti/Types.hs module
Iustin Pop [Thu, 15 Nov 2012 09:05:45 +0000 (10:05 +0100)]
Create a new Ganeti/Types.hs module

There are already three cases where we copied type definitions between
the htools-specific types into the main ganeti code. Let's stop doing
this ☺ and create a common types module that holds these.

Note that there already exists BasicTypes.hs, but that refers to very
low-level types, and can't use TH derivation itself.

A side effect of this unification is that there is a small conflict
between AdminStatus/AdminOffline and InstanceStatus/AdminOffline. As
such, I renamed AdminOffline and AdminDown to StatusOffline/StatusDown
in the InstanceStatus type.

The patch also moves the tests related to these types to a new test
module.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Adeodato Simo <dato@google.com>

8 years agolocking: Simplify condition
Michael Hanselmann [Mon, 19 Nov 2012 16:14:44 +0000 (17:14 +0100)]
locking: Simplify condition

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoconfigure: Shell cleanup
Michael Hanselmann [Wed, 14 Nov 2012 10:59:04 +0000 (11:59 +0100)]
configure: Shell cleanup

- Remote unnecessary quote characters
- Change some uses of “test "$foo"” to “test -n "$foo"”
- Merge “if/else/if” into “if/elif”
- Wrap lines longer than 80 characters and add a test to “make check”

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd opcode for running commands remotely
Michael Hanselmann [Thu, 15 Nov 2012 10:16:09 +0000 (11:16 +0100)]
Add opcode for running commands remotely

The opcode doesn't pay attention to the build-time flag to enable or
disable restricted commands. In a cluster different nodes could have
different settings.

Node locks are acquired in shared mode by default, but the use of an
exclusive lock can be forced using the “use_locking” parameter.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd RPC for restricted commands
Michael Hanselmann [Tue, 13 Nov 2012 20:11:34 +0000 (21:11 +0100)]
Add RPC for restricted commands

Expose the functionality to run restricted commands remotely via RPC.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agolocking: Don't schedule pending acq. for short timeout
Michael Hanselmann [Fri, 16 Nov 2012 16:43:50 +0000 (17:43 +0100)]
locking: Don't schedule pending acq. for short timeout

Scheduling a pending acquisition is relatively expensive and lot of code
is involved. Unless there is already one, a new pipe needs to be opened.
Data structures need to be updated as well, only to be undone shortly
after. This patch adds a small condition to return straight away if the
timeout is shorter than 1ms and the lock couldn't be acquired right
away. Blocking acquisitions are not affected, as are acquisitions with
longer timeouts.

While I tried hard, I could not find a way of writing a test
specifically for this case which wouldn't have involved making
significant other modifications. Other tests already exercise the new
code, though.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoPrevent hlint crash on unicode chars
Michele Tartara [Mon, 19 Nov 2012 09:30:20 +0000 (10:30 +0100)]
Prevent hlint crash on unicode chars

Some versions of hlint crash on the Attoparsec test file because they
are not able to correctly handle unicode characters if the proper encoding
(--utf8) is not specified.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoRename design document for SSH setup
Michael Hanselmann [Fri, 16 Nov 2012 15:38:17 +0000 (16:38 +0100)]
Rename design document for SSH setup

More design details will be added regarding setting up the node daemon,
so this is more about adding a node than just setting up SSH.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agobackend: Implement remote commands
Michael Hanselmann [Tue, 13 Nov 2012 19:07:55 +0000 (20:07 +0100)]
backend: Implement remote commands

As per design document (doc/design-remote-commands.rst), a number of
rather strict tests is applied to any incoming request, a delay is
inserted upon errors and returned error messages are very generic
(unless it's the actual command that failed). There are unit tests for
all of the newly added code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoconfigure: Add option to enable remote commands
Michael Hanselmann [Wed, 14 Nov 2012 10:23:33 +0000 (11:23 +0100)]
configure: Add option to enable remote commands

By default remote commands are disabled and need to be explicitely
enabled at build time.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd unit test for RAPI handler access definitions
Michael Hanselmann [Wed, 14 Nov 2012 01:46:01 +0000 (02:46 +0100)]
Add unit test for RAPI handler access definitions

- Ensure query-related resources have the same access permissions
  (specifically “/2/query/*” and “/2/*/console”)
- Check access permission consistency (write implies read)

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoUpdate ganeti-rapi man page for new user option
Michael Hanselmann [Thu, 8 Nov 2012 18:35:42 +0000 (19:35 +0100)]
Update ganeti-rapi man page for new user option

Instead of duplicating what is already described in the full
documentation, a reference is added instead. Man pages can't use
constants or assertions, therefore it is easier for them to get out of
sync with the code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agorapi: Add new user option for querying
Michael Hanselmann [Thu, 8 Nov 2012 18:31:14 +0000 (19:31 +0100)]
rapi: Add new user option for querying

This was requested in issue 301. Before this patch, requests to
“/2/query/*” and “/2/instances/*/console” would require authentication
with a user with write access. Since that is not strictly necessary, a
new user option named “read” is added.

Console information can also be retrieved as a normal query, therefore
the change applies there too.

This was the first user option to be added after “write”, therefore
quite a few changes were necessary. Documentation, including NEWS, is
updated as well.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoAdd Utility functions for loading data from test files
Michele Tartara [Fri, 16 Nov 2012 12:23:02 +0000 (13:23 +0100)]
Add Utility functions for loading data from test files

They mimic their python counterparts.

Added functions:
* getSourceDir
* testDataFilename
* readTestData
* readPythonTestData

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoImproved TestHelper module docstring
Michele Tartara [Fri, 16 Nov 2012 08:53:37 +0000 (09:53 +0100)]
Improved TestHelper module docstring

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoMove gnt_cluster.SHOW_MACHINE_OPT to cli
Michael Hanselmann [Thu, 15 Nov 2012 09:01:38 +0000 (10:01 +0100)]
Move gnt_cluster.SHOW_MACHINE_OPT to cli

This allows the option to be re-used in other places.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoAdd ganeti-node-role ocf example file
Guido Trotter [Tue, 8 Nov 2011 12:13:07 +0000 (12:13 +0000)]
Add ganeti-node-role ocf example file

This allows offlining nodes that don't respond if they are part of a
linux-HA cluster.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoAdd ganeti-master-role.ocf example file
Guido Trotter [Fri, 4 Nov 2011 12:46:25 +0000 (12:46 +0000)]
Add ganeti-master-role.ocf example file

This allows controlling the cluster master role if the nodes are part of
a linux-HA cluster.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoCleanup THH function use from built module namespace
Iustin Pop [Tue, 13 Nov 2012 16:27:23 +0000 (17:27 +0100)]
Cleanup THH function use from built module namespace

Currently, THH.hs "injects" into the built code names of library
functions like Text.JSON.makeObj, Ganeti.JSON.fromObj, etc. built
directly from strings, via (e.g.)

  varE (mkName "makeObj")

This means that the "makeObj" name must exist in the target module,
i.o.w. must be imported there. This leads to the strange case of
having to have imports that do not appear at all in the used
(template) code, but are needed to satisfy this "hidden" dependency;
look at Ganeti/Jobs.hs before this patch, for example.

This is also not very obvious, because we usually import Text.JSON
anyway; I only stumbled upon it while doing some cleanup work.

So to clean this up, the current patch changes the THH.hs to use not
string-derived, but identifier-derived names («'identifier» versus
«mkName "identifier"»); this is better, as the names must be
resolvable when compiling THH itself (once), and not when compiling
the multiple derived modules. As you can see, this allows removal of
extraneous imports from various modules.

Background information: an `mkName "foo"` results in a name of flavour
NameS (“An unqualified name; dynamically bound”) or alternatively to a
qualified name, but still dynamically bound. Whereas what we want is a
statically bound name: `'foo` results in a NameG flavour, “Global name
bound outside of the TH AST: An original name”.

One more explanation: the change is similar to going from 'x = eval
"map"' to 'x = map'; the name is no longer dynamically evaluated, but
statically when the module is compiled. In our case, previously names
were bound at target module compile time, now they are bound at THH.hs
compile time.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoSwitch Attoparsec/unicode test from QC to HUnit
Iustin Pop [Tue, 13 Nov 2012 15:20:34 +0000 (16:20 +0100)]
Switch Attoparsec/unicode test from QC to HUnit

This is not a quickcheck property, since it doesn't have any
variable/arbitrary inputs. So let's make it a test case, and shorten a
bit the name.

Also, sorry for not catching this in the review.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

8 years agoWarn on invalid lines in HTTP user files
Michael Hanselmann [Wed, 14 Nov 2012 01:45:46 +0000 (02:45 +0100)]
Warn on invalid lines in HTTP user files

Without this change, invalid lines or values would be silently ignored.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoDon't check for remote command directory as file storage
Michael Hanselmann [Wed, 14 Nov 2012 08:39:53 +0000 (09:39 +0100)]
Don't check for remote command directory as file storage

This test does not work properly if localstatedir is not “/etc”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoDocument master daemon's shutdown behaviour
Michael Hanselmann [Wed, 14 Nov 2012 08:02:53 +0000 (09:02 +0100)]
Document master daemon's shutdown behaviour

This wasn't clearly documented until now.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agopathutils: Add directory for remote commands
Michael Hanselmann [Mon, 5 Nov 2012 17:49:23 +0000 (18:49 +0100)]
pathutils: Add directory for remote commands

Also add tests to ensure it's never allowed as a file storage path. A
constant for the lock file is also added.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoutils.process.RunResult: Always set "fail_reason" attribute
Michael Hanselmann [Tue, 13 Nov 2012 19:11:10 +0000 (20:11 +0100)]
utils.process.RunResult: Always set "fail_reason" attribute

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd previously missing node daemon GID to getent mock
Michael Hanselmann [Tue, 13 Nov 2012 19:10:39 +0000 (20:10 +0100)]
Add previously missing node daemon GID to getent mock

The UID is there, the GID wasn't.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd test utility to count calls to function
Michael Hanselmann [Tue, 13 Nov 2012 19:10:32 +0000 (20:10 +0100)]
Add test utility to count calls to function

In some cases it's nice to verify a function has been called exactly N
times. This is going to be used in tests for remote commands.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoSort content of generated RPC code
Michael Hanselmann [Tue, 13 Nov 2012 20:18:37 +0000 (21:18 +0100)]
Sort content of generated RPC code

This is similar to commit f5ce761. A stable sort order makes changes
between versions easier to find.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoAdd new test for RAPI
Michael Hanselmann [Wed, 7 Nov 2012 16:39:50 +0000 (17:39 +0100)]
Add new test for RAPI

Unlike existing tests, this actually tests RAPI at the interface with
the HTTP server. This way authentification can also be tested. A test
for “/2/query/…” is included as it's a bit special.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

8 years agoAdd “gnt-job change-priority” sub-command
Michael Hanselmann [Wed, 24 Oct 2012 01:46:15 +0000 (03:46 +0200)]
Add “gnt-job change-priority” sub-command

This can be used to change the priority of a pending or running job (the
latter only if there are unprocessed opcodes).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

8 years agoExpose changing job priority via LUXI
Michael Hanselmann [Wed, 24 Oct 2012 01:46:01 +0000 (03:46 +0200)]
Expose changing job priority via LUXI

A new LUXI request is added, in both Python and Haskell.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agojqueue: Allow changing of job priority
Michael Hanselmann [Wed, 24 Oct 2012 01:45:23 +0000 (03:45 +0200)]
jqueue: Allow changing of job priority

This is due to a feature request. Sometimes one wants to change the
priority of a job after it has been submitted, e.g. after submitting an
important job only to later notice many other pending jobs which will be
processed first. Priority changes only take effect at the next lock
acquisition or when the job is re-scheduled.

The design is very similar to how jobs are cancelled.

Unit tests for “_QueuedJob.ChangePriority” are included.

Also rename “TestQueuedJob.test” to “TestQueuedJob.testError”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

8 years agojqueue: Set task ID for jobs added to workerpool
Michael Hanselmann [Fri, 2 Nov 2012 19:03:50 +0000 (20:03 +0100)]
jqueue: Set task ID for jobs added to workerpool

The job ID is re-used as the task ID, as job IDs are unique.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

8 years agoworkerpool: Preserve task number when deferring
Michael Hanselmann [Thu, 1 Nov 2012 18:06:02 +0000 (19:06 +0100)]
workerpool: Preserve task number when deferring

When a task is deferred it should receive the same task ID upon being
returned to the pool.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

8 years agoworkerpool: Add method to change task's priority
Michael Hanselmann [Tue, 30 Oct 2012 15:44:02 +0000 (16:44 +0100)]
workerpool: Add method to change task's priority

Using the task ID a pending task's priority can be changed. This will be
used to change the priority of jobs in the workerpool.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

8 years agoworkerpool: Change data structure for priority change
Michael Hanselmann [Tue, 30 Oct 2012 15:44:27 +0000 (16:44 +0100)]
workerpool: Change data structure for priority change

To prepare for the addition of a new function allowing changing a
pending task's priority, the internal data structure is slightly
changed. The (optional) task ID is stored as part of the task entry. A
new dictionary provides a mapping from the task ID to its task entry. If
the task ID is None, the entry is not added to the map.

Task entries used to be a tuple, but since modifying the priority
requires changing an entry, they are changed to lists in this patch.
Tuple items can not be modified.

The underlying idea is from [1].

[1]:
http://docs.python.org/library/heapq.html#priority-queue-implementation-notes

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>

8 years agoAutomatically enable version-dependent GHC flags
Iustin Pop [Tue, 13 Nov 2012 10:46:04 +0000 (11:46 +0100)]
Automatically enable version-dependent GHC flags

Some GHC flags are very useful, but only appear in more recent GHC
versions. To support the use of such flags while still supporting
older compilers, let's add conditional checks and enabling based on
the results.

Currently only `-fwarn-incomplete-uni-patterns` is enabled, which
detects refutable patterns in lambda expressions and pattern bindings
(e.g. "let (Just x) = y"); such constructs are bad as they can lead to
runtime exceptions.

Additionally, fix an existing such bad construct in a test case; we
workaround it by using error, since that should never fail.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoAdded attoparsec unit test for Unicode parsing
Michele Tartara [Tue, 13 Nov 2012 11:31:13 +0000 (12:31 +0100)]
Added attoparsec unit test for Unicode parsing

Attoparsec is known to have had issues with parsing non-ASCII strings.
This test makes sure that parsing of Unicode characters works fine.

Signed-off-by: Michele Tartara <mtartara@google.com>
[iustin: small doc string fixes]
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd "Attoparsec" to the optional haskell packages
Michele Tartara [Fri, 9 Nov 2012 15:46:05 +0000 (16:46 +0100)]
Add "Attoparsec" to the optional haskell packages

This will be needed for the data collectors of the monitoring agent.

* Detection of the library
* Creation of the appropriate variables
* Update to the installation documentation

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoDocumentation for the NODE_RES level
Helga Velroyen [Fri, 9 Nov 2012 13:52:50 +0000 (14:52 +0100)]
Documentation for the NODE_RES level

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoImprove make regen-vcs-version
Iustin Pop [Mon, 12 Nov 2012 16:35:53 +0000 (17:35 +0100)]
Improve make regen-vcs-version

"make regen-vcs-version" is run at every ./devel/upload time, in order
to have reasonably up-to-date information in the uploaded
scripts/binaries. However, this means it currently triggers
recompilation of (at least) Ganeti/Version.hs, and sometimes more than
that.

Since we only care about the contents of the file and not the
timestamp, let's change the make rule so that it only updates the file
if there are actually changes in the content, as checked by
"cmp". This results in a much faster ./devel/upload, especially for
small changes in the (non-Haskell) code, or when the code has already
been compiled.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoFix two hlint warnings
Iustin Pop [Mon, 12 Nov 2012 16:35:32 +0000 (17:35 +0100)]
Fix two hlint warnings

Sorry, I broke lint again :), by introducing two sub-standard changes.

Additionally, this silences an older existing warning that only
triggers with some versions of hlint (e.g. 1.8.28 which is present in
Wheezy).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoRunCmd: Expose "postfork" callback
Michael Hanselmann [Fri, 9 Nov 2012 15:12:22 +0000 (16:12 +0100)]
RunCmd: Expose "postfork" callback

The “_postfork_fn” parameter was only used for tests until now. To
implement a good locking scheme, remote commands must also make use of
this callback to release a lock when the command was successfully
started (but did not yet finish).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoMerge branch 'devel-2.6' into master
Iustin Pop [Mon, 12 Nov 2012 16:11:17 +0000 (17:11 +0100)]
Merge branch 'devel-2.6' into master

* devel-2.6:
  Improve error message when migration status fail
  Fix type error in kvm/GetMigrationStatus
  Fix PID file writing in Haskell daemons

Conflicts (both trivial):
        htools/Ganeti/Daemon.hs (_writePidFile rename)
        lib/hypervisor/hv_kvm.py (master fixed an indentation issue)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoMerge branch 'stable-2.6' into devel-2.6
Iustin Pop [Mon, 12 Nov 2012 14:13:39 +0000 (15:13 +0100)]
Merge branch 'stable-2.6' into devel-2.6

* stable-2.6:
  Improve error message when migration status fail
  Fix type error in kvm/GetMigrationStatus
  Fix PID file writing in Haskell daemons

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoImprove error message when migration status fail
Iustin Pop [Mon, 12 Nov 2012 11:57:58 +0000 (12:57 +0100)]
Improve error message when migration status fail

Commit 6a1434d7 (“Make migration RPC non-blocking”) changed the API
for reporting migration status, but has a small cosmetic bug: if the
migration status if failure, but the RPC itself to get the status
didn't fail, it shows the following error message:

  Could not migrate instance instance2: None

since it always uses result.fail_msg, irrespective of which part of
the if condition failed.

This patch simply updates the msg if not already set, leading to:

  Could not migrate instance instance2: hypervisor returned failure

Proper error display can be done once the migration status objects can
return failure information as well, beside status.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

8 years agoFix type error in kvm/GetMigrationStatus
Iustin Pop [Mon, 12 Nov 2012 11:50:52 +0000 (12:50 +0100)]
Fix type error in kvm/GetMigrationStatus

Commit 6a1434d7 (“Make migration RPC non-blocking”) changed from
raising HypervisorErrors to returning MigrationStatus
objects. However, these objects don't have an "info" attribute, so
they can't pass a reason back (which is in itself a bug); but the KVM
hypervisor code attempts to do so, and fails at runtime with:

  Failed to get migration status: 'MigrationStatus' object has no attribute 'info'

instead of the intended:

  Migration failed, aborting: too many broken 'info migrate' answers

For now (on stable-2.6), let's just remove the "info" reason, and
later we can add it back properly once we have a way to correctly
represent migration status failures in the LU.

This fixes issue 297.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoFix PID file writing in Haskell daemons
Iustin Pop [Mon, 12 Nov 2012 11:14:37 +0000 (12:14 +0100)]
Fix PID file writing in Haskell daemons

Currently, the code uses createFile, which has the effect of always
truncating the file. This is bad, as the content of the PID file is
wiped even when we wouldn't be able to lock it!

We switch to openFd (createFile is just a wrapper over that), and we
use an explicit set of flags; defaultFileFlags is already safe
(trunc=False), but I prefer to set it explicitly with our desired
flags.

Note that this bug doesn't manifest in normal usage, as daemon-util
won't try to start the daemon if already running. But if anyone or
anything does call ganeti-confd explicitly, the pid file will be
emptied and the daemon will keep trying to be restarted forever…

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoRemove unsafePerformIO usage
Iustin Pop [Thu, 8 Nov 2012 17:52:31 +0000 (18:52 +0100)]
Remove unsafePerformIO usage

We need to change a few things, most importantly CLI options defaults,
but otherwise we already used the path to files from functions which
were already in the I/O monad, so we don't have to change much of the
code flow.

Additionally, Path.hs now has an explicit export list, to stop
accidental leakage of symbols.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

8 years agoChange keyToFilename/readSSconfFile interaction
Iustin Pop [Thu, 8 Nov 2012 17:48:19 +0000 (18:48 +0100)]
Change keyToFilename/readSSconfFile interaction

Currently, keyToFilename uses itself the default config path. In the
light of making that an function in the IO monad, let's remove the
"default" path functionality from this function and make it always
require the config path; its caller, readSSConfFile, can then do the
maybe/default path change.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoChange type of program options to 'IO [Options]'
Iustin Pop [Thu, 8 Nov 2012 14:04:10 +0000 (15:04 +0100)]
Change type of program options to 'IO [Options]'

Some options have defaults that depend on the environment, and we
could handle these in two ways:

- use a place-holder value (e.g. data X a = Default | Custom a) that
  is later read from the environment
- move the options list to IO monad, where it can read the
  environment, etc.

The second option allows also displaying the actual defaults in the
`--help' output, even though it's not as nice, so I went with it.

This patch only changes the option types, without actually changing
any options yet.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

8 years agoTwo small logging improvements
Iustin Pop [Tue, 6 Nov 2012 14:14:55 +0000 (15:14 +0100)]
Two small logging improvements

The "starting" message is changed to match the Python one, and in case
the preparation fails, we also log the error (beside printing it on
stderr or writing it to the error reporting pipe), as at this time
logging is usually set up.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoAnnotate confd startup/luxi binding error
Iustin Pop [Tue, 6 Nov 2012 14:06:52 +0000 (15:06 +0100)]
Annotate confd startup/luxi binding error

The luxi binding is now annotated for better readability.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoSwitch writing of PID file to new reporting style
Iustin Pop [Tue, 6 Nov 2012 13:53:45 +0000 (14:53 +0100)]
Switch writing of PID file to new reporting style

Currently, the writing of the PID file uses a "standard" error
reporting: catch exception, transform it into a 'Result' type, leave
handling of that to the caller. However, for daemon startup, we
actually want exceptions to be propagated up until the handler which
will write the details to the pipe.

This patch removes the writePidFile wrapper (and stops exporting it),
and changes the code to simply annotate better the I/O error.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoMove and rename BasicTypes.annotateIOError
Iustin Pop [Tue, 6 Nov 2012 13:49:53 +0000 (14:49 +0100)]
Move and rename BasicTypes.annotateIOError

It turns out that annotateIOError already exists in the standard
library, with a different purpose (this made me waste 10 minutes
trying to understand why the types were wrong…). Let's move this out
of BasicTypes, since it's used only in Daemon.hs, and also rename it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoSwitch daemon startup to pipe-based error reporting
Iustin Pop [Fri, 2 Nov 2012 16:01:30 +0000 (17:01 +0100)]
Switch daemon startup to pipe-based error reporting

This changes the daemon reporting error to the same mechanism as in
Python: use a pipe which is written to from the forked children.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoMove preparation steps of confd in prepMain
Iustin Pop [Fri, 2 Nov 2012 13:37:47 +0000 (14:37 +0100)]
Move preparation steps of confd in prepMain

This does the address parsing earlier, before we fork, for
better/faster error reporting.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoSplit queryd run into prepare and exec
Iustin Pop [Thu, 1 Nov 2012 14:38:53 +0000 (15:38 +0100)]
Split queryd run into prepare and exec

This will help with the general daemon split of prepare/run, and flag
errors earlier in the startup.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoConvert tag objects to a safer type
Iustin Pop [Thu, 8 Nov 2012 12:51:11 +0000 (13:51 +0100)]
Convert tag objects to a safer type

Currently, we keep information about the "target" of a tag operation
in a data type similar to (TagKind, Maybe String). This is unsafe, as
nothing (at the type level) prevents us from accidentally having
(TagCluster, Just "instance1.example.com"), or (TagInstance, Nothing).

To fix this problem, we rename the current TagObject type to TagType
(an internal utility type), and create TagObject as a better/safer
data type (see the definition), which doesn't allow such possibilities
in the future.

The downside is that, since at encoding level (both opcode and luxi)
this is done in an ugly way (type elements spread at the same level as
level as other value), we have to add custom encoders/decoders. The
encoder is shared between the OpCode and Luxi usage, the decoder is
different however as Luxi uses custom decoding.

This also fixes the recent breakage in confd w.r.t. QueryTags.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoSwitch Luxi TH code from simple to custom fields
Iustin Pop [Thu, 8 Nov 2012 12:26:40 +0000 (13:26 +0100)]
Switch Luxi TH code from simple to custom fields

This is needed so that we have more flexibility in generating Luxi
serialisation code (deserialisation is still custom). Also, only
exceptions are now using the 'simple' field types, so we might be able
later to convert and remove that TH code as well.

Since we will use custom serialisation fields in the future, we change
the order of serialisation for custom-save fields; Luxi uses
positional as opposed to name-based ordering, so we need to keep this
stable.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

8 years agoFix display of results in ==? operator
Iustin Pop [Thu, 8 Nov 2012 12:19:58 +0000 (13:19 +0100)]
Fix display of results in ==? operator

Doh, this is exactly the opposite of what we wanted… good thing no
test failed so far :)

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoMake THH:genSaveOpCode a bit more general
Iustin Pop [Thu, 8 Nov 2012 12:08:32 +0000 (13:08 +0100)]
Make THH:genSaveOpCode a bit more general

This can be improved, by taking all hardcoded names as parameters, to
serve as a more-general "build save clause for a multi-constructor
data type". I'm not renaming the function as well, since I don't know
exactly how much we can abstract later.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

8 years agosphinx_ext: Allow use of “rapi” module in pyeval
Michael Hanselmann [Thu, 8 Nov 2012 18:08:07 +0000 (19:08 +0100)]
sphinx_ext: Allow use of “rapi” module in pyeval

This way constants like “rapi.RAPI_ACCESS_WRITE” can be used in
documentation.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agorlib2: Document two previously undocumented functions
Michael Hanselmann [Thu, 8 Nov 2012 17:51:03 +0000 (18:51 +0100)]
rlib2: Document two previously undocumented functions

Commit 208a6cff just included empty docstrings.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agojqueue/mcpu: Determine priority using callback
Michael Hanselmann [Wed, 24 Oct 2012 01:05:05 +0000 (03:05 +0200)]
jqueue/mcpu: Determine priority using callback

Instead of being given the priority for acquiring locks by means of a
parameter, mcpu will now call back. This is in preparation for
implementing a command to change a job's priority on the fly and allows
to change it while locks are being acquired (taking effect on the next
lock acquire).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>