nbma.git
8 years agoUpdate git submodule to 2.1.2.1 master
Guido Trotter [Wed, 12 May 2010 15:23:38 +0000 (16:23 +0100)]
Update git submodule to 2.1.2.1

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Luca Bigliardi <shammash@google.com>

8 years agoUpdate HMAC_CLUSTER_KEY to the new Ganeti name
Mike Jurney [Tue, 11 May 2010 22:10:17 +0000 (18:10 -0400)]
Update HMAC_CLUSTER_KEY to the new Ganeti name

HMAC_CLUSTER_KEY was changed to CONFD_HMAC_KEY in the main ganeti
constants.py, but not yet in the NBMA repository.

Signed-off-by: Mike Jurney <jurney@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoRemove source network based ip rule
Guido Trotter [Tue, 23 Mar 2010 22:45:59 +0000 (22:45 +0000)]
Remove source network based ip rule

Since ganeti inserts rules which are source interface based since
2.1.0~rc0, and we depend on 2.1.1 for nbma anyway, we don't need the
source address based one anymore.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

8 years agoFix path mtu discovery from the instances
Guido Trotter [Tue, 23 Mar 2010 21:56:17 +0000 (21:56 +0000)]
Fix path mtu discovery from the instances

When policy routing is used, we want icmp fragmentation-needed packets
to go directly to the instances via their direct route, rather than be
routed according to our main routing table. This iptables rule sets a
local netfilter mark on the packets that forces them to be looked up
against the separate routing table.

This initial fix requires the instance networks to be non overlapping.

Since only the fragmentation needed packets are fwmarked any other
communication from the node to the instance and vice versa is still
forbidden, through the direct path.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

8 years agoAdd fwmark based ip rule
Guido Trotter [Tue, 23 Mar 2010 22:52:25 +0000 (22:52 +0000)]
Add fwmark based ip rule

On nbma nodes we establish the convention that packets marked with a
table number, will be redirected to that routing table. This will be
useful to mark specific packets that have to reach the instances.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

8 years agorun-in-tempdir: use system ganeti if available
Guido Trotter [Wed, 24 Mar 2010 11:55:49 +0000 (11:55 +0000)]
run-in-tempdir: use system ganeti if available

If the ganeti is installed on the local system, we don't need to copy
its libraries to the tmpdir, because it will be picked up.

On the other hand, if we need to install our copy, we also need to build
it, or it won't be working (missing _autoconf.py).

This fixes "make check".

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

8 years agocheck-ganeti-version: handle import error
Guido Trotter [Wed, 24 Mar 2010 11:54:15 +0000 (11:54 +0000)]
check-ganeti-version: handle import error

If the ganeti module is not found at all, we will make
check-ganeti-version exit nicely, rather than with a traceback. Although
this won't happen during ./configure, because the module presence is
checked by AC_PYTHON_MODULE, we plan to use this script for more
functionality.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

8 years agoDon't inject values in the ganeti constants
Guido Trotter [Tue, 23 Mar 2010 18:08:28 +0000 (18:08 +0000)]
Don't inject values in the ganeti constants

Since Ganeti 2.1.1 our constants are already in Ganeti

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

8 years agoRequire ganeti version 2.1.1 or above
Guido Trotter [Tue, 23 Mar 2010 18:07:47 +0000 (18:07 +0000)]
Require ganeti version 2.1.1 or above

Document the requirement in NEWS, and verify it in configure

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

8 years agoRevert "Add Ganeti submodule version check to configure"
Guido Trotter [Tue, 23 Mar 2010 18:04:35 +0000 (18:04 +0000)]
Revert "Add Ganeti submodule version check to configure"

This reverts commit d29ca524856ff6ad7e1fd81be4d30e7ada720476.
This commit was never LGTMed. In fact it was NACKed.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

8 years agoDisable "method could be a function" lint message
Guido Trotter [Tue, 23 Mar 2010 17:57:27 +0000 (17:57 +0000)]
Disable "method could be a function" lint message

R0201: 91:NLDRequestProcessor._Ping: Method could be a function

Yes, it could, but there's no harm in leaving it as is, to make it more
similar to the other queries.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Balazs Lecz <leczb@google.com>

8 years agoImplement route invalidate request handling
Balazs Lecz [Tue, 23 Feb 2010 12:18:07 +0000 (12:18 +0000)]
Implement route invalidate request handling

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoAdd Ganeti submodule version check to configure
Balazs Lecz [Thu, 18 Mar 2010 18:06:18 +0000 (18:06 +0000)]
Add Ganeti submodule version check to configure

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoAdd basis for NLD->NLD communication
Balazs Lecz [Wed, 17 Feb 2010 16:59:53 +0000 (16:59 +0000)]
Add basis for NLD->NLD communication

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoMove nld->confd related code into a separate file
Balazs Lecz [Tue, 16 Feb 2010 19:22:48 +0000 (19:22 +0000)]
Move nld->confd related code into a separate file

This is to clean up daemons/ganeti-nld, which started to bloat.

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoLook up source node of misrouted packets
Balazs Lecz [Fri, 12 Feb 2010 11:31:20 +0000 (11:31 +0000)]
Look up source node of misrouted packets

After an instance migration or failover some nodes will
have stale routing information and these will continue to send traffic
to the old node. This is detected by ganeti-nld on the old node.

This patch adds the capability to look up the source node based on the
source instance IP.

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoFactor out the instance-node map
Balazs Lecz [Thu, 11 Feb 2010 13:37:03 +0000 (13:37 +0000)]
Factor out the instance-node map

Moving it from the NLDCondCallback class (up one level)
into the NetworkLookupDaemon class.

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoAdd misrouted packet handler stub
Balazs Lecz [Tue, 9 Feb 2010 19:01:06 +0000 (19:01 +0000)]
Add misrouted packet handler stub

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoAdd NFLOG iptables initialization
Balazs Lecz [Tue, 9 Feb 2010 18:26:40 +0000 (18:26 +0000)]
Add NFLOG iptables initialization

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoAdd stub for NFLOG support
Balazs Lecz [Tue, 9 Feb 2010 12:35:26 +0000 (12:35 +0000)]
Add stub for NFLOG support

Also add a missing step to run-in-tempdir that copies the Ganeti library
into the test tempdir.

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoAdd nflog as a checked dependency
Balazs Lecz [Fri, 5 Mar 2010 10:02:41 +0000 (10:02 +0000)]
Add nflog as a checked dependency

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoUpdate ganeti.git submodule to aeb5cab
Balazs Lecz [Mon, 8 Mar 2010 14:06:36 +0000 (14:06 +0000)]
Update ganeti.git submodule to aeb5cab

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoDistribute missing files
Guido Trotter [Mon, 8 Mar 2010 17:36:53 +0000 (17:36 +0000)]
Distribute missing files

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoAdd check-nld watcher hook
Guido Trotter [Fri, 26 Feb 2010 12:56:24 +0000 (13:56 +0100)]
Add check-nld watcher hook

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoMove the NLD constant to... constants! :)
Guido Trotter [Fri, 26 Feb 2010 12:49:16 +0000 (13:49 +0100)]
Move the NLD constant to... constants! :)

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoStart using the MASTER_NBMA_INTERFACE config
Balazs Lecz [Mon, 8 Feb 2010 12:12:30 +0000 (12:12 +0000)]
Start using the MASTER_NBMA_INTERFACE config

Signed-off-by: Balazs Lecz <leczb@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoImplement parsing of per-cluster options
Balazs Lecz [Wed, 10 Feb 2010 16:53:12 +0000 (16:53 +0000)]
Implement parsing of per-cluster options

Signed-off-by: Balazs Lecz <leczb@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoInstantiate multiple periodic updaters
Balazs Lecz [Thu, 4 Feb 2010 16:36:12 +0000 (16:36 +0000)]
Instantiate multiple periodic updaters

Also make it clear what "name" means by renaming it to "cluster_name".

Signed-off-by: Balazs Lecz <leczb@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoAdd per-cluster config file handler stub
Balazs Lecz [Wed, 3 Feb 2010 18:11:55 +0000 (18:11 +0000)]
Add per-cluster config file handler stub

This adds the cluster config file keys and adds a "default" cluster
populated with default configuration values.

Signed-off-by: Balazs Lecz <leczb@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoAdd per-cluster configuration file
Guido Trotter [Wed, 3 Feb 2010 12:44:49 +0000 (12:44 +0000)]
Add per-cluster configuration file

This file can be used on the endpoints to specify per-cluster data.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

8 years agoBump up version to 0.2.0 v0.2.0
Guido Trotter [Thu, 4 Feb 2010 14:34:03 +0000 (14:34 +0000)]
Bump up version to 0.2.0

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoAdd NEWS for versions 0.1.0 and 0.2.0
Guido Trotter [Thu, 4 Feb 2010 14:33:49 +0000 (14:33 +0000)]
Add NEWS for versions 0.1.0 and 0.2.0

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoAdd initial devnotes file
Guido Trotter [Thu, 4 Feb 2010 14:20:13 +0000 (14:20 +0000)]
Add initial devnotes file

For now we're mentioning the git submodule, which people might not be
used to. Additionally we should probably say something about build
dependencies and such, but that probably would come after some user
notes.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoRemove unnecessary pylint disable
Balazs Lecz [Tue, 2 Feb 2010 15:00:05 +0000 (15:00 +0000)]
Remove unnecessary pylint disable

Since the addition of the Ganeti pylintrc this is not required.

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>

8 years agoLog error and empty answers at warning level
Balazs Lecz [Tue, 2 Feb 2010 14:51:34 +0000 (14:51 +0000)]
Log error and empty answers at warning level

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>

8 years agoAdd periodic master node IP update
Balazs Lecz [Wed, 27 Jan 2010 21:01:04 +0000 (21:01 +0000)]
Add periodic master node IP update

Fetch the master IP and the primary IP for the master node from the Ganeti
cluster via confd and route it through the NBMA network.
This allows the cluster to stay reachable, even when a master failover
takes place (provided the master IP is within the NBMA network range).

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>

8 years agoUpdate gitignore with a few recent additions
Guido Trotter [Tue, 2 Feb 2010 11:32:20 +0000 (11:32 +0000)]
Update gitignore with a few recent additions

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoFix ganeti.confd.client import
Balazs Lecz [Tue, 2 Feb 2010 11:30:44 +0000 (11:30 +0000)]
Fix ganeti.confd.client import

It was reported as unused by pylint but *was* used.

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>

8 years agoBuild devel/upload
Guido Trotter [Mon, 1 Feb 2010 16:41:41 +0000 (16:41 +0000)]
Build devel/upload

This cannot just be symlinked, as it needs some configure variables to
be replaced.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoLink devel/review from the ganeti.git submodule
Guido Trotter [Mon, 1 Feb 2010 16:36:26 +0000 (16:36 +0000)]
Link devel/review from the ganeti.git submodule

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd lint makefile target
Guido Trotter [Mon, 1 Feb 2010 16:15:23 +0000 (16:15 +0000)]
Add lint makefile target

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoAdd pylint check to ./configure
Guido Trotter [Mon, 1 Feb 2010 16:11:09 +0000 (16:11 +0000)]
Add pylint check to ./configure

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoAdd ganeti and ganeti_nbma makefile targets
Guido Trotter [Mon, 1 Feb 2010 15:58:53 +0000 (15:58 +0000)]
Add ganeti and ganeti_nbma makefile targets

These create symlinks for the two python modules

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoSymlink some common files from ganeti.git
Guido Trotter [Mon, 1 Feb 2010 15:47:05 +0000 (15:47 +0000)]
Symlink some common files from ganeti.git

- replace the autotools/ac_python_module.m4 copy with a symlink
- symlink the pylintrc configuration file

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

8 years agoAdd ganeti as an nbma submodule
Guido Trotter [Mon, 1 Feb 2010 15:44:37 +0000 (15:44 +0000)]
Add ganeti as an nbma submodule

We use ganeti.git as a path, as we want to leave "ganeti" for the python
libraries. The submodule is initialized at 2.1.0~rc5, on stable-2.1.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoFix pylint warnings in test/nbma.config_unittest.py
Balazs Lecz [Thu, 28 Jan 2010 17:24:14 +0000 (17:24 +0000)]
Fix pylint warnings in test/nbma.config_unittest.py

Fixed this:
 - ************* Module nbma.config_unittest
 - F0401: 27: Unable to import 'nbma'

Silenced these:
 - ************* Module nbma.config_unittest
 - C0103: 32:TestBashFragmentConfigParser.setUp: Invalid name "setUp" (should match (_{0,2}[A-Z]+[a-z0-9]+([A-Z]+[a-z0-9]*)*|__.*__|([a-z_][a-z0-9_]*))$)
 - C0103: 36:TestBashFragmentConfigParser._testParser: Invalid name "_testParser" (should match (_{0,2}[A-Z]+[a-z0-9]+([A-Z]+[a-z0-9]*)*|__.*__|([a-z_][a-z0-9_]*))$)
 - C0103: 55:TestBashFragmentConfigParser.testStringLoad: Invalid name "testStringLoad" (should match (_{0,2}[A-Z]+[a-z0-9]+([A-Z]+[a-z0-9]*)*|__.*__|([a-z_][a-z0-9_]*))$)
 - C0103: 59:TestBashFragmentConfigParser.testNonDefaultSection: Invalid name "testNonDefaultSection" (should match (_{0,2}[A-Z]+[a-z0-9]+([A-Z]+[a-z0-9]*)*|__.*__|([a-z_][a-z0-9_]*))$)
 - C0103: 64:TestBashFragmentConfigParser.testFileLoad: Invalid name "testFileLoad" (should match (_{0,2}[A-Z]+[a-z0-9]+([A-Z]+[a-z0-9]*)*|__.*__|([a-z_][a-z0-9_]*))$)
 - C0103: 68:TestBashFragmentConfigParser.testFileLoadSection: Invalid name "testFileLoadSection" (should match (_{0,2}[A-Z]+[a-z0-9]+([A-Z]+[a-z0-9]*)*|__.*__|([a-z_][a-z0-9_]*))$)

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>

8 years agoFix pylint warnings in lib/*.py
Balazs Lecz [Thu, 28 Jan 2010 14:07:11 +0000 (14:07 +0000)]
Fix pylint warnings in lib/*.py

Fixed these:
 - ************* Module lib
 - C0111:  1: Missing docstring
 - ************* Module lib.config
 - C0111:  1: Missing docstring
 - C0322: 30: Operator not preceded by a space
 - DEFAULT_SECTION="default"
 -                ^
 - C0322: 31: Operator not preceded by a space
 - ENDPOINT_EXTIP_KEY="endpoint_external_ip"
 -                   ^
 - C0322: 32: Operator not preceded by a space
 - INTERFACE_KEY="gre_interface"
 -              ^
 - C0322: 33: Operator not preceded by a space
 - TABLE_KEY="routing_table"
 -          ^
 - W0622: 47:BashFragmentConfigParser.LoadFragmentFromString: Redefining built-in 'str'
 - E0602:115:NLDConfig.FromConfigFiles: Undefined variable 'constants'
 - E0602:122:NLDConfig.FromConfigFiles: Undefined variable 'constants'
 - E0602:135:NLDConfig.FromConfigFiles: Undefined variable 'constants'
 - E0602:136:NLDConfig.FromConfigFiles: Undefined variable 'constants'
 - ************* Module lib.constants
 - C0111:  1: Missing docstring
 - C0322: 27: Operator not preceded by a space
 - DEFAULT_ROUTING_TABLE="100"
 -                      ^
 - C0322: 28: Operator not preceded by a space
 - DEFAULT_NEIGHBOUR_INTERFACE="gtun0"
 -                            ^
 - ************* Module lib.iptables
 - W0622: 51:_GenRandomString: Redefining built-in 'len'
 - C0324:128:UpdateIptablesRules: Comma not followed by a space
 -   new_ips = "%s%s" % (ips_prefix,ips_suffix)
 -                                 ^^
 - ************* Module lib.server
 - W6501: 60:PeerSetManager._UpdateIptablesRules: Specify string format arguments as logging function parameters
 - W0631: 61:PeerSetManager._UpdateIptablesRules: Using possibly undefined loop variable 'peer_list'
   |
   \--> this one uncovered a bug. Fixed it.

Silenced these:
 - ************* Module lib.iptables
 - W0402: 38: Uses of a deprecated module 'string'
 - W0612: 92:CheckIptablesChain: Unused variable 'dest'

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>

8 years agoFix pylint warnings in daemons/ganeti-nld
Balazs Lecz [Wed, 27 Jan 2010 18:11:10 +0000 (18:11 +0000)]
Fix pylint warnings in daemons/ganeti-nld

Fixed these:
 - C:123:NLDConfdCallback.UpdateInstanceIPList: Operator not preceded by a space
 -     mapping_query={
 -                  ^
 -         gnt_constants.CONFD_REQQ_IPLIST: iplist,
 -         gnt_constants.CONFD_REQQ_LINK: link,
 - R:115:NLDConfdCallback.UpdateInstanceIPList: Method could be a function
 - W:146:NLDConfdCallback.UpdateInstanceNodeMapping: Specify string format arguments as logging function parameters
 - W:150:NLDConfdCallback.UpdateInstanceNodeMapping: Specify string format arguments as logging function parameters
 - W:172:NLDConfdCallback.__call__: Specify string format arguments as logging function parameters
 - W:180:NLDConfdCallback.__call__: Specify string format arguments as logging function parameters
 - C:218:NLDPeriodicUpdater._EnableTimers: Missing docstring
 - W:329:NetworkLookupDaemon.ExecNld: Unused variable 'updater'
 - W: 41: Unused import iptables
 - W: 36: Unused import asyncore
 - W: 53: Unused import ganeti

Disabled these:
 - C:  1: Invalid name "ganeti-nld" (should match (([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$)
 - W:179:NLDConfdCallback.__call__: Unused variable 'err'
 - R:188:NLDPeriodicUpdater.__init__: Too many arguments (7/5)
 - W:307:NetworkLookupDaemon.CheckNld: Attribute 'config' defined outside __init__
 - W:318:NetworkLookupDaemon.ExecNld: Unused argument 'args'

Remaining:
 - W:328: FIXME: Create a unique string per cluster, when we'll have more than one

Signed-off-by: Balazs Lecz <leczb@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>

8 years agoFix __slots__ definition in the NLDConfig class
Balazs Lecz [Mon, 25 Jan 2010 20:35:41 +0000 (20:35 +0000)]
Fix __slots__ definition in the NLDConfig class

According to http://docs.python.org/reference/datamodel.html

* The action of a __slots__ declaration is limited to the class where it
  is defined. As a result, subclasses will have a __dict__ unless they
  also define __slots__ (which must only contain names of any
  **additional** slots).

* If a class defines a slot also defined in a base class, the instance
  variable defined by the base class slot is inaccessible (except by
  retrieving its descriptor directly from the base class). This renders
  the meaning of the program undefined. In the future, a check may be
  added to prevent this.

Signed-off-by: Balazs Lecz <leczb@google.com>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd ac_python_module.m4 to nbma autotools
Guido Trotter [Mon, 25 Jan 2010 11:02:14 +0000 (11:02 +0000)]
Add ac_python_module.m4 to nbma autotools

This is needed to work with older versions of automake, which don't have
this functionality included.

Signed-off-by: Guido Trotter <ultrotter@google.com>

8 years agoAdd debugging statements for nld callback
Guido Trotter [Thu, 14 Jan 2010 15:22:06 +0000 (15:22 +0000)]
Add debugging statements for nld callback

After removing spurious updates it's hard to debug if a server is
receiving responses, after the first one, or not. Adding a debug
statement for each reply received.

Also write a couple of missing docstrings.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoganeti-nld: use PeerSetManager
Guido Trotter [Thu, 10 Dec 2009 17:29:51 +0000 (17:29 +0000)]
ganeti-nld: use PeerSetManager

We currently have only one cluster supported, so using the
PeerSetManager doesn't change much. The only difference is that the
endpoints list is added as a peer set.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoAdd PeerSetManager class
Guido Trotter [Thu, 10 Dec 2009 16:37:11 +0000 (16:37 +0000)]
Add PeerSetManager class

In order to support multiple remote clusters, we merge the nbma peer
list in a common place. This code has been written in a new server
module in order to try to make the main ganeti-nld daemon smaller, and
use module code.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

8 years agoDon't force spurious table entries in the config
Guido Trotter [Thu, 10 Dec 2009 15:39:12 +0000 (15:39 +0000)]
Don't force spurious table entries in the config

Currently if a config file has no table or interface, the default
mapping is pushed to the table, even if there is an association in a
further file. Fix this by pushing an association only if a config file
has either a table or an interface (and using defaults, if not both, to
be compatible with what we did before). The default mapping, at the end,
is only pushed if no other associations are found.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

9 years agoganeti-nld: fix a cut&paste error in a docstring
Guido Trotter [Wed, 18 Nov 2009 13:59:12 +0000 (13:59 +0000)]
ganeti-nld: fix a cut&paste error in a docstring

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

9 years agoFix two indentation issues
Guido Trotter [Wed, 9 Dec 2009 15:15:39 +0000 (15:15 +0000)]
Fix two indentation issues

They were noted by iustin in the CL commit but I forgot to fix them
before submitting.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

9 years agoganeti-nld: avoid spurious updates
Guido Trotter [Tue, 8 Dec 2009 14:52:46 +0000 (14:52 +0000)]
ganeti-nld: avoid spurious updates

If values haven't changed since we last queried them, there's no need to
update them in the system. Add some caches and only perform the updates
when there's a real value change.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

9 years agoganeti-nld: collapse mapping query
Guido Trotter [Tue, 8 Dec 2009 14:30:18 +0000 (14:30 +0000)]
ganeti-nld: collapse mapping query

Rather than doing one query per instance, make only one query in total.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

9 years agonld: unhardcode "gtun0"
Guido Trotter [Mon, 16 Nov 2009 16:08:02 +0000 (16:08 +0000)]
nld: unhardcode "gtun0"

Currently we have a hardcoded constant in nld for the target tunnel
interface. Rather than doing that, we can send one instance group
request per routing table we're aware of, and then depending on the
routing table use the correct tunnel, according to the config files.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

9 years agoMove confd client inside NLDPeriodicUpdater
Guido Trotter [Mon, 16 Nov 2009 15:46:04 +0000 (15:46 +0000)]
Move confd client inside NLDPeriodicUpdater

Small cleanup, that also allows the full configuration to be accessed by
the periodic updater.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

9 years agoNLDConfig: parse tables/links as well
Guido Trotter [Wed, 30 Sep 2009 12:21:17 +0000 (13:21 +0100)]
NLDConfig: parse tables/links as well

We want nld to know what the correct mapping between the routing table
number (used by ganeti as a routed interface link) and physical nbma
devices is.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

9 years agoAdd default networking constants
Guido Trotter [Wed, 30 Sep 2009 12:26:25 +0000 (13:26 +0100)]
Add default networking constants

These represent our default routing table and its associated neighbour
interface, used when config files don't specify a mapping.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

9 years agoUpdateNodeIPList: don't change confd answer
Guido Trotter [Tue, 8 Dec 2009 11:57:57 +0000 (11:57 +0000)]
UpdateNodeIPList: don't change confd answer

We shouldn't change the confd answer, because if we do we break the
cache. Create a local copy and append only to that one.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

9 years agoganeti-nld: Add a way to update the mc file
Guido Trotter [Mon, 16 Nov 2009 15:09:18 +0000 (15:09 +0000)]
ganeti-nld: Add a way to update the mc file

If asked on the command line we'll update the mc list file with fresh
data from the cluster

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoNLDConfdCallback: filter and warn on error results
Guido Trotter [Mon, 16 Nov 2009 16:56:10 +0000 (16:56 +0000)]
NLDConfdCallback: filter and warn on error results

If we get an error result from a query, warn rather than try to send it
to the callback.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoganeti-nld: update MC list periodically
Guido Trotter [Mon, 16 Nov 2009 12:15:32 +0000 (12:15 +0000)]
ganeti-nld: update MC list periodically

Every once in a while (120s is ok, we don't expect lots of MCs to be
changing most of the time, and we're safe as long as we have some
overlap) update the MC list, making sure we can still talk to the
cluster even if the MCs change. In the future we should also save the
startup file, but just on non-nodes.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoganeti-nld: fix initial startup
Guido Trotter [Mon, 16 Nov 2009 12:12:52 +0000 (12:12 +0000)]
ganeti-nld: fix initial startup

When we start nld we want to perform the queries immediately. Calling
Update* is broken, though, because each Update function will think it's
being called as a result of a timer action, and thus disable the current
timer and enable a new one, which schedules one timer per update
function.

In order to solve this we just call _EnableTimers, but pass in a special
parameter to make sure the timer is set to 0 for the first run.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoBump up version to 0.1.0 v0.1.0
Guido Trotter [Fri, 23 Oct 2009 20:00:42 +0000 (16:00 -0400)]
Bump up version to 0.1.0

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Tom Limoncelli <tlim@google.com>

9 years agoAdd "proto static" to the routing setup routes
Guido Trotter [Fri, 23 Oct 2009 19:55:28 +0000 (15:55 -0400)]
Add "proto static" to the routing setup routes

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Tom Limoncelli <tlim@google.com>

9 years agoUnittest BashFragmentConfigParser for parenthesis
Guido Trotter [Thu, 22 Oct 2009 01:53:49 +0000 (21:53 -0400)]
Unittest BashFragmentConfigParser for parenthesis

Make sure the behavior is expected. This also merges the string + file
test to generate the string using ReadFile.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoChange routeback to be per-network
Guido Trotter [Thu, 22 Oct 2009 01:19:26 +0000 (21:19 -0400)]
Change routeback to be per-network

Rather than only having the possibility of routing traffic via the nbma
or not, we can selectively choose which networks to route back and which
to reach directly, through our default route.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoAllow spaces before the end of line in ip rules
Guido Trotter [Wed, 21 Oct 2009 13:56:50 +0000 (09:56 -0400)]
Allow spaces before the end of line in ip rules

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoUse "onlink" rather than adding/removing routes
Guido Trotter [Tue, 20 Oct 2009 17:32:17 +0000 (13:32 -0400)]
Use "onlink" rather than adding/removing routes

According to Atis Elsts' suggestion on the linux-netdev list
(http://marc.info/?l=linux-netdev&m=125605724732608&w=2) this allows us
to specify a route forcing the kernel to consider the gateway local to
the link.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoMake the AM_ENDPOINT grep more secure
Guido Trotter [Tue, 20 Oct 2009 11:11:37 +0000 (07:11 -0400)]
Make the AM_ENDPOINT grep more secure

Currently it would match partial IP addresses. Require that the ip
starts with a space, and is followed by either a space or a / (space is
used for pointopoint, / when there is a suffix.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agorouting_setup: make grep safer
Guido Trotter [Tue, 20 Oct 2009 09:46:01 +0000 (05:46 -0400)]
routing_setup: make grep safer

Also, avoid spawning a new shell to execute the ip rule command, and use
the return value of grep instead.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agorouting_setup.in: use builtin test
Guido Trotter [Mon, 19 Oct 2009 23:11:52 +0000 (19:11 -0400)]
routing_setup.in: use builtin test

Use [[ rather than [ in a couple of if tests.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoMake sure instance traffic is routed back in nbma
Guido Trotter [Fri, 16 Oct 2009 22:22:08 +0000 (18:22 -0400)]
Make sure instance traffic is routed back in nbma

If ROUTEBACK is selected in the config file, we force all instance
traffic to be routed back through the NBMA, rather than allowing it to
go back directly.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoip addr add: specify a netmask to the address
Guido Trotter [Sat, 17 Oct 2009 23:18:14 +0000 (19:18 -0400)]
ip addr add: specify a netmask to the address

If we don't do that the "default" one according to the ip address we're
using is going to be set.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agorouting setup: use route replace
Guido Trotter [Fri, 16 Oct 2009 22:20:37 +0000 (18:20 -0400)]
routing setup: use route replace

Rather than route add it's better to use route replace, so to override
any existing one. Also we put the table at the beginning of the line to
increase readability (replace WHERE WHAT to WHAT).

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoNLD: add endpoints to allowed nodes
Guido Trotter [Fri, 16 Oct 2009 15:00:18 +0000 (11:00 -0400)]
NLD: add endpoints to allowed nodes

- Pass the nld config to the callback
- Use the config "endpoint" slot to extend the node list, passed to the
  iptables rules generator
- Profit

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoNLD: load config files passed in as arguments
Guido Trotter [Fri, 16 Oct 2009 14:50:30 +0000 (10:50 -0400)]
NLD: load config files passed in as arguments

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoAdd package sysconf dir to python constants
Guido Trotter [Wed, 30 Sep 2009 10:02:14 +0000 (11:02 +0100)]
Add package sysconf dir to python constants

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoconfig: NLDConfig class
Guido Trotter [Wed, 30 Sep 2009 12:21:17 +0000 (13:21 +0100)]
config: NLDConfig class

This class holds configuration for NLD, loading it from shell fragments.

For now it contains an endpoints list.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoconfig BashFragmentConfigParser
Guido Trotter [Wed, 30 Sep 2009 10:49:37 +0000 (11:49 +0100)]
config BashFragmentConfigParser

This class converts a bash fragment into an ini file and imports it
through the python config parser.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoNLD: move startup of NLD into its own class
Guido Trotter [Wed, 30 Sep 2009 11:27:31 +0000 (12:27 +0100)]
NLD: move startup of NLD into its own class

Move the CheckNld and ExecNld functions into a class, so check can
perform some config loading/checking and pass it to exec if successful.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoAdd basic unittesting framework
Guido Trotter [Wed, 7 Oct 2009 11:51:05 +0000 (12:51 +0100)]
Add basic unittesting framework

For now no test files are shipped, but the basic framework is added to
the makefile. The framework is mostly copied/adapted from Ganeti core.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoiptables_setup: selectively disable conntrack
Guido Trotter [Fri, 16 Oct 2009 12:51:10 +0000 (08:51 -0400)]
iptables_setup: selectively disable conntrack

We don't want to pollute conntrack tables with packets to/from the
instance network, or gre packets (since they might be distributed anyway
over different endpoints). So we disable conntracking for them.

This patch also moves standard setup to be done only if the setup script
is called without an explicit config file, so that the conntracking
rulers can be applied separately at each call.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoRevert "iptables_setup: don't load config file"
Guido Trotter [Fri, 16 Oct 2009 12:23:48 +0000 (08:23 -0400)]
Revert "iptables_setup: don't load config file"

This reverts commit 07a19582d2d330e436eb27ae5184e636f823d7ea.
We'll be using information from the config file in iptables as well.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoSplit the config file per endpoint
Guido Trotter [Wed, 30 Sep 2009 10:00:06 +0000 (11:00 +0100)]
Split the config file per endpoint

Since each node can have multiple endpoints, we load one common config
file, but optionally we accept another one, so the scripts can be run
multiple times, one per endpoint.

At the same time move configuration to our own @sysconfdir@/ganeti/nbma
directory, rather than polluting the ganeti space.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoiptables_setup: don't load config file
Guido Trotter [Wed, 30 Sep 2009 10:15:05 +0000 (11:15 +0100)]
iptables_setup: don't load config file

No configuration dependent variable is used, currently, while setting
up iptables.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agogre_setup: load the ip_gre module if not present
Guido Trotter [Wed, 30 Sep 2009 08:40:36 +0000 (09:40 +0100)]
gre_setup: load the ip_gre module if not present

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoInitial ganeti-nld code
Guido Trotter [Fri, 25 Sep 2009 15:17:07 +0000 (16:17 +0100)]
Initial ganeti-nld code

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoCreate python constants and _autoconf
Guido Trotter [Mon, 28 Sep 2009 15:24:23 +0000 (16:24 +0100)]
Create python constants and _autoconf

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoDistribute the nbma python package
Guido Trotter [Mon, 28 Sep 2009 15:14:05 +0000 (16:14 +0100)]
Distribute the nbma python package

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoChange project name to ganeti-nbma
Guido Trotter [Mon, 28 Sep 2009 13:24:40 +0000 (14:24 +0100)]
Change project name to ganeti-nbma

At the same time move the scripts libdir to @LIBDIR@/ganeti/nbma, and
the config file to @SYSCONFDIR@/ganeti/nbma.conf for consistency.

Moreover we modify dist_doc_DATA not to distribute the example config
file, as it shouldn't be shipped in the root documentation dir.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

9 years agorouting_setup: enable more forwarding on endpoints
Guido Trotter [Mon, 28 Sep 2009 10:55:16 +0000 (11:55 +0100)]
routing_setup: enable more forwarding on endpoints

If we only forward on certain interfaces, rather than all, make sure
that, for the endpoint, we also include the endpoint netdev.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agogre_setup: configure endpoint ip or mapping
Guido Trotter [Mon, 28 Sep 2009 10:06:53 +0000 (11:06 +0100)]
gre_setup: configure endpoint ip or mapping

If we are on the endpoint, configure the internal ip at tunnel setup
time. Otherwise setup a neigh entry mapping to reach the endpoint.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agocommon.sh: add automatic endpoint detection
Guido Trotter [Mon, 28 Sep 2009 10:48:03 +0000 (11:48 +0100)]
common.sh: add automatic endpoint detection

With this the various setup scripts can behave slightly different on
corp endpoints. The network device to check for "endpointness" can be
specified.

As a start we force using the main routing table on endpoints.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoMove config file load into common.sh fragment
Guido Trotter [Mon, 28 Sep 2009 10:41:41 +0000 (11:41 +0100)]
Move config file load into common.sh fragment

Rather than checking/loading the config file in all scripts, we move the
loading in one common place. As a bonus we also provide some default
values, and check for the config pieces that cannot be missing because
they are site/cluster-specific.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

9 years agoChange BASE_NETDEV to GRE_BASE_NETDEV
Guido Trotter [Mon, 28 Sep 2009 11:04:35 +0000 (12:04 +0100)]
Change BASE_NETDEV to GRE_BASE_NETDEV

So it's clearly only the base netdev on which the gre tunnel insists.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>