/posts/ Recent content in Articles on The Mole's OpenNMS (TM) Dev Diary Hugo -- gohugo.io en-us Thu, 02 Mar 2017 12:00:00 -0500 /posts/development/1-how-to-compile/ Sun, 19 May 2019 13:29:25 +0100 /posts/development/1-how-to-compile/ This article describes how to compile OpenNMS from source. If you have not set up your environment properly, please see here. Checkout the Code First of all, the code should be checked out. I usually do this in my ~/dev directory: git clone https://github.com/OpenNMS/opennms.git This will create an opennms directory in ~/dev/opennms. Next cd into it: cd opennms. Build from Source TL;DR Regular development time (. /posts/development/0-dev-environment-macos/ Sat, 18 May 2019 10:30:00 +0100 /posts/development/0-dev-environment-macos/ Here we cover what is necessary to build OpenNMS from Source and afterwards start it locally on your Mac OS system. Compile Prerequisites In order to compile OpenNMS at least the following requirements should be met: The latest JDK 8 version should be installed makensis cloned github OpenNMS repository maven (optional) Run Prerequisites PostgreSQL server Jicmp, jicmp6 (both optional) /posts/development/11-opennms-inside-karaf-first-try/ Tue, 12 Mar 2019 10:00:00 +0100 /posts/development/11-opennms-inside-karaf-first-try/ The other day I found my self in the lucky position to upgrade Karaf to a newer version. While doing that I also wrote something about our Http Bridge and why it was not be the best decision to integrate Karaf inside Jetty instead of the other way around. While implementing the Upgrade I asked my self how hard could it be to get some components of now running from within Jetty to running in Karaf. /posts/karaf-upgrade/4.2.2/day14/ Thu, 07 Mar 2019 15:00:00 +0100 /posts/karaf-upgrade/4.2.2/day14/ Yesterday we encountered the issue, that the docker image build by Bamboo was not able to run on the agent itself, but locally did. As with docker this should not happen, so we have to investigate. It turns out, that some of our JAXB dependencies we put into ${karaf.home}/lib fail while loading. Simply removing them solved at least the Smoke Test failures. The last test failing, is probably a flapper. /posts/karaf-upgrade/4.2.2/day13/ Wed, 06 Mar 2019 15:00:00 +0100 /posts/karaf-upgrade/4.2.2/day13/ Today I spend half of the day documenting the adventure of Updating Karaf to Version 4.2.2/4.2.3. Besides that a bunch of KarafTestCaseITs were still failing. As the issue is not solvable for now I decided to @Ignore them. Another issue is, that some of the system tests, e.g. HealthCheckIT fail on Bamboo, but pass locally on my system. Usually if something fails on Bamboo it can be reproduced locally and then fixed. /posts/karaf-upgrade/4.2.2/day12/ Tue, 05 Mar 2019 17:00:00 +0100 /posts/karaf-upgrade/4.2.2/day12/ I thought with all the issues I already fixed, now everything should work again. NOPE. Bamboo is telling me that NxosTelemetryIT is no longer working. Okay, that is easy to reproduce with my system-test.sh script, inspired by some version from Jesse White. Running the script with the Bamboo URL to the build, e.g. /system-tests.sh https://bamboo.opennms.org/browse/OPENNMS-ONMS2773-14 as an argument will download the RPMs and build the same docker images as Bamboo would. /posts/karaf-upgrade/4.2.2/day11/ Mon, 04 Mar 2019 17:00:00 +0100 /posts/karaf-upgrade/4.2.2/day11/ Until now we have fixed most of our issues. However one of the remaining issues were, that locally I was not able to login to the Karaf Shell for all the docker containers of Minion/Sentinel/OpenNMS. I thought that this was the same issue as reported by Bamboo. But it turned out, that it was a local Docker on Mac VM issue. Resetting the Docker VM solved at least that problem. /posts/karaf-upgrade/4.2.2/day10/ Thu, 28 Feb 2019 17:00:00 +0100 /posts/karaf-upgrade/4.2.2/day10/ Yesterday we spend all day fiddling with our KarafTestCases and finally disabled them. Today I thought we were going to fix some of our failed Smoke Tests, but instead we are going to do something different. Looking at the docker containers build for our System Tests, I encountered the following exception in the karaf.log. Caused by: java.lang.IllegalArgumentException: Not supported: http://javax.xml.XMLConstants/property/accessExternalDTD Consulting the Oracle pointed me to a topic on the Karaf Mailing List. /posts/karaf-upgrade/4.2.2/day9/ Wed, 27 Feb 2019 12:00:00 +0100 /posts/karaf-upgrade/4.2.2/day9/ Yesterday we fixed all Vaadin related issue and were very positive to have fixed a lot of issues and pushed our changes to Bamboo to see what actually all is broken. Integration Test wise, all Karaf related tests failed. I investigated the issue and long story short: Hacking the startup.properties in our KarafTestCase is no longer working, instead the library must be provided otherwise. Executing the tests don’t work anymore and result in a ClassNotFoundException, bundle wiring is no longer valid /posts/karaf-upgrade/4.2.2/day8/ Tue, 26 Feb 2019 12:00:00 +0100 /posts/karaf-upgrade/4.2.2/day8/ Yesterday we learned, that Vaadin UIs are no longer working as they should. The sympthoms were that the first Vaadin UI you visited were always presented to you, no matter which one you accessed afterwards. My first finding was, that this is caused by the way Vaadin implements the @PreserveOnRefresh functionality. See NMS-10601 for more details. However after I addressed/fixed that issue, the problem was not. Debugging some more showed, that only one OsgiUIProvider was present in the VaadinSession, but multiple should be. /posts/karaf-upgrade/4.2.2/day7/ Mon, 25 Feb 2019 12:00:00 +0100 /posts/karaf-upgrade/4.2.2/day7/ Last week I started replacing our forked Http Bridge with a more OSGi friendly and generic approach. I got to a point where it somewhat worked, but Vaadin UIs seem to have a problem with the new structure. So today I am going to investigate the issue some more. Debugging the issue showed that the widgetset property, required for Vaadin UIs is not propagated properly, resulting in loading the DefaultWidgetSet which - for the Topology UI - is missing certain components. /posts/karaf-upgrade/4.2.2/day6/ Thu, 21 Feb 2019 12:00:00 +0100 /posts/karaf-upgrade/4.2.2/day6/ Yesterday we started to to replace our custom Http Bridge with the official Http Felix Bridge. I started implementing a ProxyFilter to dispatch all requests to the OSGi world if it can be handled by any OSGi registered service to make /opennms/topology work instead of using /opennms/osgi/topology. While finalizing this initial work, I encountered a problem that service properties supported by the PAX WEB project are not officially supported by OSGi and with that not supported by the Apache Felix HTTP project. /posts/karaf-upgrade/4.2.2/day5/ Wed, 20 Feb 2019 12:00:00 +0100 /posts/karaf-upgrade/4.2.2/day5/ Yesterday I found out, that we can no longer use our forked Http Service Bridge. This is good for once, as we can now finally get rid of it. But on the other hand this, will be a lot of work. So let’s get started. Which version to use As we can no longer use the pax-whiteboard bundle as we used to, we have to find another solution. I asked the Oracle and found out, that the Apache HTTP Felix project provides a " /posts/karaf-upgrade/4.2.2/day4/ Tue, 19 Feb 2019 12:00:00 +0100 /posts/karaf-upgrade/4.2.2/day4/ Yesterday I learned, that the pax-whiteboard bundle is no longer starting. Investigating the issue some further revealed, that when activating the bundle an Exception java.lang.IllegalStateException: HttpService must be implementing Pax-Web WebContainer! is thrown. This is a known issue (with OpenNMS), but before Karaf 4.2.2 it just worked. My guess is, that the Exception was triggered AFTER the bundle was started. However now this happens when the bundle is still starting and therefore failing. /posts/karaf-upgrade/4.2.2/day3/ Mon, 18 Feb 2019 12:00:00 +0100 /posts/karaf-upgrade/4.2.2/day3/ Last week I started upgrading Karaf to version 4.2.2. You can see the endevour started at Day 1 and continued at Day 2. At Day 2 we left off with three exception in the karaf.log. While two were something like "Cannot register an already registered Mbean" which are most likely caused by some code (inside Karaf) trying to register already registered Mbeans by Jetty itself. So the "real" Exception to investigate here is the issue with the plugin manager. /posts/development/10-versions/ Fri, 15 Feb 2019 15:00:00 +0100 /posts/development/10-versions/ I always find myself wondering which Horizon Version a Meridian release is based on. To solve this problem once and for all, I looked them all up by looking at the pom.xml of each foundation* branch. Which Horizon Version is Meridian based on? Meridian Horizon Branch M2015 H14 foundation M2016 H17 foundation-2016 M2017 /posts/karaf-upgrade/4.2.2/day2/ Thu, 14 Feb 2019 18:00:00 +0100 /posts/karaf-upgrade/4.2.2/day2/ While yesterday I managed to fix One exception, a bunch of more showed up. I wanted to investigate those isuses today, but found a weird behaviour: After a restart of OpenNMS the web console was no longer available. Besides that, I got a ClassNotFoundException org.eclipse.jetty.jaas.JAASLoginService in the logs. First thing I did was to add the dependency <dependency> <groupId>org.eclipse.jetty</groupId> <artifactId>jetty-jaas</artifactId> <version>${jettyVersion}</version> </dependency> to OpenNMS and copy it to the lib directory. /posts/karaf-upgrade/4.2.2/day1/ Wed, 13 Feb 2019 17:00:00 +0100 /posts/karaf-upgrade/4.2.2/day1/ We are attempting to get OpenNMS to run on Java 9 . The first step of this is to make it compile for Java 9, which we are working on actively . However making it run/compile on Java 9 requires a lot of dependencies to be upgrades as they are not Java 9 compatible as well. One of those dependency upgrades is to upgrade Apache Karaf to version 4.2.2, as you can see in the following figure. /posts/development/9-hindenburg-effect/ Wed, 13 Feb 2019 15:00:00 +0100 /posts/development/9-hindenburg-effect/ In the very past OpenNMS was mostly driven by Jetty and Spring*. Probably a more flexible solution needed to be found, as with a big application like OpenNMS dependency issues are always present. For example dependency A requires dependency B in Version 1, but dependency C requires the same dependency B in Version 2. Modern containers - such as Apache Karaf (an OSGi container) - can solve these problems. Besides that OSGi offers a nice API and Implementation differentiation. /posts/bootstrap-4-migration/post-2/ Wed, 13 Feb 2019 14:00:00 +0100 /posts/bootstrap-4-migration/post-2/ The Bootstrap 4 Migration is about to be finalized. When we showed the work to the community (and also around internally) the feedback was mostly positive. Some pages did not render properly, but besides that the most commonly reported issues were: Card Headers use up too much space Elements use up too much space in general Concerns about the blue as this is supposed to be reserved for OpenNMS Meridian /posts/development/8-dont-export-the-same-package/ Tue, 29 Jan 2019 15:30:00 +0100 /posts/development/8-dont-export-the-same-package/ Today I wanted to wire in the new GraphService into OpenNMS and have a graph:list command show a GraphML graph I sent to the /opennms/rest/graphml endpoint. The idea was to leverage the GraphmlRepository to also store a configuration file for a new GraphMLGraphContainerProviderServiceFactory implementing a ManagedServiceFactory. Basically the same as we already do with the GraphMLTopologyFactory I wired everything together and was also able to install the features. However as soon as I send the graphml document to the rest endpoint via (of course OpenNMS should be started) /posts/graph-service/4-hibernate-where-issue/ Mon, 28 Jan 2019 14:30:00 +0100 /posts/graph-service/4-hibernate-where-issue/ While working on the persistence for the new graph service I encountered a weird behaviour: Each time a persisted graph is altered the edges were removed. So I investigated and it turns out if you are using hibernate inheritance in combination with the @Where clause hibernate only persists the entities which have been modified and removes the relation to the others, but keeps the entities sigh. For more clarity, here is the original code /posts/development/6-show-config-from-karaf-shell/ Sat, 26 Jan 2019 09:00:00 +0100 /posts/development/6-show-config-from-karaf-shell/ Sometimes it is required to show a (bundle) configuration from the Karaf Shell. This may be on OpenNMS, Minion or Sentinel. First of all a connection to the Karaf Shell must be made. Here is described on how to do this. The according ports can be found here. Now let’s assume we want to see the current values of the id, location and other settings on Minion. First of all we need to know in which cfg file the property is located. /posts/development/7-default-ssh-ports/ Sat, 26 Jan 2019 08:00:00 +0100 /posts/development/7-default-ssh-ports/ Here I explained on how to connect to the OpenNMS Karaf Shell. However when connecting to Sentinel or Minion the ports are different. The following table shows the default ports for each system System Port OpenNMS 8101 Minion 8201 Sentinel 8301 /posts/development/4-connect-to-karaf-shell/ Wed, 23 Jan 2019 13:00:00 +0100 /posts/development/4-connect-to-karaf-shell/ OpenNMS uses Apache Karaf under the hood and in some cases (especially during development) it is required to connect to the OpenNMS Karaf Shell. First of all OpenNMS must be started to connect to the Karaf Shell. I usually start OpenNMS with the -v (verbose) and -t (debug) options: $OPENNMS_HOME/bin/opennms -vt start Afterwards I check with $OPENNMS_HOME/bin/opennms -v status until the Jetty Webserver is STARTED, as the Karaf Container is started by the Webapp itself. /posts/development/3-graphml-tooltip/ Tue, 22 Jan 2019 13:00:00 +0100 /posts/development/3-graphml-tooltip/ One user in the community is using the GraphML Topology and asked me in chat if it were possible to add tooltips to a vertex from within a groovy script. Short answer: Yes Long answer By default, setting the tooltip is supposed to work only by setting it directly in the graphml file itself. The official documentation for this says, that the name of the attribute is tooltipText. So if you are looking for static tooltips, that is the way to go. /posts/graph-service/2-moving-forward/ Mon, 21 Jan 2019 15:00:00 +0100 /posts/graph-service/2-moving-forward/ With the new POC implementation we could solve the current problems of the Topology in OpenNMS. The next step is to take what was learned and implement something similar and more robust in OpenNMS. As this is a big chunk to swallow, the main issue here is on how to split the work into small pieces, as with this kind of API you basically have to almost implement everything before it can be used. /posts/graph-service/1-poc/ Sun, 20 Jan 2019 14:00:00 +0100 /posts/graph-service/1-poc/ The Topology Map in OpenNMS has some architectural problems. Mainly that there is no "service layer" implemented, which causes mostly performance issues. Besides this no API or persistence model is enforced. This prevents easy integration with 3rd party applications or even provide a new UI implementation. Therefore I started playing around with a new Topology implementation (See issue HZN-1452) for more details). In order to not confuse it with existing implementations, it is called Graph Service or Graph Engine here. /posts/bootstrap-4-migration/post-1/ Sun, 20 Jan 2019 13:29:25 +0100 /posts/bootstrap-4-migration/post-1/ In OpenNMS we use Bootstrap 3 as a library to help styling web pages. However the style sheets are customized in a way, that they are not bootstrap-compatible anymore. Figure 1. Bootstrap 3 Default Theme Figure 2. OpenNMS customized Bootstrap 3 Theme As you can see, the custom stylings do not follow the bootstrap "schema", so each component is either not fully provided (e.g. list stylings or panels), or they are using a different context (The OpenNMS color’s indicate the severity context, whereas the bootstrap *-info, *-warning, etc. /posts/my-first-post/ Sun, 20 Jan 2019 13:29:25 +0100 /posts/my-first-post/ public class Main { public static void main(String[] args) { System.out.println("Hello World!"); } }