ARTICLES

Upgrading Karaf to 4.2.2 - Day 11

Until now we have fixed most of our issues. However one of the remaining issues were, that locally I was not able to login to the Karaf Shell for all the docker containers of Minion/Sentinel/OpenNMS.

I thought that this was the same issue as reported by Bamboo. But it turned out, that it was a local Docker on Mac VM issue. Resetting the Docker VM solved at least that problem.

The next issue I encountered is the following

Error resolving artifact org.opennms.integration.api:api:jar:1.0.0-alpha6-SNAPSHOT

This was also reported by indigo.

It was pretty simple to find the cause of this. Looking into the system directory revealed the following:

% find system -type f -iname "*alpha6*.jar"
system/org/opennms/integration/api/xml/1.0.0-alpha6-SNAPSHOT/xml-1.0.0-alpha6-20190220.141239-1.jar
system/org/opennms/integration/api/common/1.0.0-alpha6-SNAPSHOT/common-1.0.0-alpha6-20190220.141235-1.jar
system/org/opennms/integration/api/api/1.0.0-alpha6-SNAPSHOT/api-1.0.0-alpha6-20190220.141241-1.jar

Looking in the maven repository the situation looks like this

% find  ~/.m2/repository/org/opennms/integration/api/api/1.0.0-alpha6-SNAPSHOT -type f -iname "*.jar"
/Users/mvrueden/.m2/repository/org/opennms/integration/api/api/1.0.0-alpha6-SNAPSHOT/api-1.0.0-alpha6-20190220.141241-1.jar
/Users/mvrueden/.m2/repository/org/opennms/integration/api/api/1.0.0-alpha6-SNAPSHOT/api-1.0.0-alpha6-SNAPSHOT.jar

So clearly the Maven Resolver cannot resolve a SNAPSHOT dependency, if it is a SNAPSHOT-Release. I replaced the concrete snapshot version (e.g. 20190220.141241-1) with SNAPSHOT and suddenly Karaf was able to resolve the bundles/artifacts.

Let’s see what the current behaviour in develop is:

% find system -type f -iname "*alpha6*.jar"
system/org/opennms/integration/api/xml/1.0.0-alpha6-SNAPSHOT/xml-1.0.0-alpha6-SNAPSHOT.jar
system/org/opennms/integration/api/common/1.0.0-alpha6-SNAPSHOT/common-1.0.0-alpha6-SNAPSHOT.jar
system/org/opennms/integration/api/api/1.0.0-alpha6-SNAPSHOT/api-1.0.0-alpha6-SNAPSHOT.jar

Okay, this should work. While working on this, indigo did not yet report the same issue, so I was assuming this is caused by the Karaf 4.2.3 Upgrade. So I reverted the karaf-feature-plugin to use version 4.1.5 instead. However I was not reliably able to reproduce the problem.

I then isolated the problem in a clean project, just creating a repository/system directory for the feature opennms-api-layer. The relevant pom.xml looked as following:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.opennms.dummy</groupId>
    <artifactId>karaf-distribution</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <properties>
        <karafVersion>4.2.3</karafVersion>
        <opennmsVersion>24.0.0-SNAPSHOT</opennmsVersion>
        <osgiJaxRsVersion>1.0.0-ONMS</osgiJaxRsVersion>
    </properties>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.karaf.tooling</groupId>
                <artifactId>karaf-maven-plugin</artifactId>
                <version>${karafVersion}</version>
                <executions>
                    <execution>
                        <id>features-add-to-repository</id>
                        <phase>process-resources</phase>
                        <goals>
                            <goal>features-add-to-repository</goal>
                        </goals>
                        <configuration>
                            <descriptors>
                                <!-- OpenNMS product features generated by container/features/pom.xml -->
                                <descriptor>mvn:org.opennms.karaf/opennms/${opennmsVersion}/xml/features</descriptor>
                            </descriptors>
                            <features>
                                <feature>opennms-api-layer</feature>
                            </features>
                            <repository>${project.build.directory}/opennms-repo</repository>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
    <dependencies>
        <dependency>
            <groupId>org.opennms.karaf</groupId>
            <artifactId>opennms</artifactId>
            <classifier>features</classifier>
            <version>${opennmsVersion}</version>
            <type>xml</type>
        </dependency>
    </dependencies>

    <!-- Required for com.eclipsesource.jaxrs/* -->
    <repositories>
        <repository>
            <snapshots><enabled>false</enabled></snapshots>
            <releases><enabled>true</enabled></releases>
            <id>opennms-repo</id>
            <name>OpenNMS Maven Repository</name>
            <url>http://maven.opennms.org/content/groups/opennms.org-release</url>
        </repository>
        <repository>
            <snapshots><enabled>true</enabled></snapshots>
            <releases><enabled>false</enabled></releases>
            <id>opennms-snapshots</id>
            <name>OpenNMS Snapshot Maven Repository</name>
            <url>http://maven.opennms.org/content/groups/opennms.org-snapshot</url>
        </repository>
    </repositories>
</project>

The generated system directory contained the -SNAPSHOT version of the api-layer artifacts.

Hmm, that is weird.

I then copied over ALL of the pom.xml from OpenNMS, fixed it, so it will compile and verified the system directory again.

And indeed the concrete snapshot versions were downloaded instead of the generic -SNAPSHOT.

Investigating and fiddling with it some more revealed that the problem was actually caused by the repositories definitions. Besides the default OpenNMS Maven Repositories we also added the default Sonatype OSS Snapshot Repository where the OpenNMS Integration API Snapshots are deployed to. However our OpenNMS Maven Repository is proxying that repository as well. This means Maven has multiple places where to download the artifacts. Looking at the Sonatype OSS Snapshot Repository there is no -SNAPSHOT artifact, but in our OpenNMS Maven Repository there is. Somehow the Sonatype OSS Snapshot Repository always wins, when there are two places and the "wrong" artifacts are beeing downloaded. Applying this patch resolved the issue.