Compare commits

...

43 Commits

Author SHA1 Message Date
Luke Powlett
8fd8184445 increment release in makefile 2019-08-08 11:05:22 +01:00
Luke Powlett
6c6c7fe1e3 release tags in fat manifest 2019-08-08 11:05:22 +01:00
Luke Powlett
684a2d1b04 specify base image as arg 2019-08-02 09:27:32 +01:00
Luke Powlett
851ede7bd9 build image from specified UBI base tag 2019-08-02 09:27:32 +01:00
LPowlett
b8a7167c4e MQ V9.1.3.0 (#348)
* MQ 9.1.3 changes

* Use crtmqdir -a

* Allow generation of TLS certificate with given hostname

* Remove check for certificate env variable

* Updated manifests and changelog for 913

* Use MQ externals to configure console frame ancestors

* Create /run/mqm

* Go sec fixes

* Set SAN when generating certificates

* Remove image source and commit
2019-07-24 12:01:21 +01:00
LPowlett
dc00231ec1 changelog for v912-UBI (#344)
* changelog for v912-UBI
2019-06-25 10:27:40 +01:00
Rob Parker
8a8ea6c6a9 Merge pull request #336 from mamgainp/windows
Windows
2019-06-13 09:39:10 +01:00
mamgainp
9714d0d513 fixes ifelse ladder 2019-06-12 15:39:41 +01:00
Paras Mamgain
6691438fa1 fixes DOWNLOAD_DIR variable for windows 2019-06-11 15:13:10 +01:00
Paras Mamgain
426c6c3b9a fixes DOWNLOAD_DIR variable for windows 2019-06-11 13:59:38 +01:00
Paras Mamgain
8088cb2902 fixes DOWNLOAD_DIR variable for windows 2019-06-11 13:59:22 +01:00
Rob Parker
a2ce23aa96 Merge pull request #334 from parrobe/capifix
use capicmd for renaming certs
2019-06-10 10:58:24 +01:00
Arthur Barr
95ba16cdb2 Merge branch 'master' into capifix 2019-06-10 10:30:50 +01:00
Rob Parker
83fe77b222 Labels and Licenses for RedHat Certification (#329)
* Add required red hat labels

* Add licenses into correct location for RedHat

* Remove reference to CIP

* PR Comments
2019-06-10 10:29:51 +01:00
Robert Parker
7ab30723cd use capicmd for renaming certs 2019-06-07 16:15:42 +01:00
Stephen Marshall
3f9fc0eaa5 Updates to resolve issues identified by gosec (#330)
* Updates to resolve issues identified by gosec
2019-06-07 11:51:34 +01:00
Arthur Barr
ee4351e55d Remove custom RPM notices file 2019-06-04 15:08:24 +01:00
Rob Parker
ebf55608d7 Merge pull request #324 from parrobe/pwfix
Set password length to meet VA scan requirements
2019-05-31 13:11:40 +01:00
Robert Parker
b3fd5f7562 typos 2019-05-31 11:47:02 +01:00
Robert Parker
3c9ec5f14c Add fix for OIDC error 2019-05-31 11:36:04 +01:00
Robert Parker
81c0b70a6f Set password length to meet VA scan requirements 2019-05-31 10:21:14 +01:00
Robert Parker
a6f307c6b5 Remove openssl 2019-05-30 16:53:27 +01:00
Robert Parker
6f677e2a59 Remove old Integration TLS function 2019-05-30 15:24:48 +01:00
Rob Parker
612fe3a9ec Merge pull request #319 from parrobe/tls
Add TLS Support
2019-05-30 10:40:51 +01:00
Robert Parker
0a9c745d96 PR comments 2019-05-30 10:09:24 +01:00
Robert Parker
b64c060ef4 Refactor TLS code 2019-05-30 09:16:40 +01:00
Robert Parker
8e22763f16 Add TLS Support 2019-05-30 09:16:40 +01:00
Stephen Marshall
40b64e620e Change image tag to amd64 2019-05-30 09:15:45 +01:00
LPowlett
700cc53c07 Handle /var/mqm/ permissions in upgrade to ubi (#316)
* handle /var/mqm/ permissions in upgrade to ubi
2019-05-29 11:40:19 +01:00
Arthur Barr
44d75b169c Remove old icon from README 2019-05-22 16:21:04 +01:00
Rob Parker
c0bf371b9e Merge pull request #314 from parrobe/makefix
Fix makefile targets
2019-05-17 12:53:24 +01:00
Robert Parker
e362644a55 Fix makefile targets 2019-05-17 08:29:32 +01:00
Stephen Marshall
c079c1b60d Merge CIP code 2019-05-16 08:58:02 +01:00
Arthur Barr
4a3bdf3b53 Containerized build 2019-05-10 09:24:36 +01:00
Arthur Barr
5ff269d2e3 Use Universal Base Image 2019-05-10 09:24:36 +01:00
Stephen Marshall
3fb2d3fe61 Fix liveness probe for miqm (#308)
* Fix liveness probe for miqm
2019-05-02 10:16:42 +01:00
Stephen Marshall
6c72c894f7 add multi-instance Queue Managers (#307)
* Initial code to implement multi-instance queue manager

* alter default mqsc to prevent race between listeners on standby startup

* Updates to multi-instance queue manager code

* initial multi instance test

* Multi-instance code improvements

* Multi instance fixes and first test

* configure queue manager

* Add mirror log filtering for mult-instance QMs

* Add log message for multi-instance enabled

* Improvements to container runtime logging

* refactor test

* Test active standby switch

* Improve createQueueManager function

* Test multi instance race

* wait

* multi instance mount tests

* skip race test

* mount tests

* no mount test

* single instance split mount tests

* readiness check

* More updates for handling standby queue manager

* Improve standby checks

* Minor fixes to miqm

* Fix logging of JSON errors

* Update copyrights

* Fix log includes
2019-05-01 14:42:25 +01:00
Rob Parker
63af43f19d Expand redaction system to handle uncommon MQSC cases (#304)
* Expand redaction system to handle uncommon MQSC cases

* add invalidMQSC fixes

* address PR comments
2019-04-17 10:34:29 +01:00
Stephen Marshall
64bb5aed8a Remove debug information 2019-04-12 13:15:02 +01:00
Rob Parker
f05a7d3eaf Merge pull request #300 from sdmarshall79/redact-mqsc-more
Handle multiple-spaces when redacting MQSC output
2019-04-09 15:43:49 +01:00
Stephen Marshall
723fe2b998 Handle multiple-spaces when redacting MQSC output 2019-04-09 15:06:25 +01:00
Stephen Marshall
44d0e0a432 Redact sensitive MQSC logs 2019-04-08 08:49:06 +01:00
LPowlett
33defc0fc9 Handle large mqsc files (#295)
* buffer mqsc input to runmqsc

* imports

* error handling on runmqsc
2019-04-04 16:46:07 +01:00
101 changed files with 5617 additions and 1503 deletions

View File

@@ -61,7 +61,8 @@ before_script:
- make deps
- echo -en 'travis_fold:end:deps\\r'
- echo 'Building Developer image...' && echo -en 'travis_fold:start:build-devserver\\r'
- make build-devserver
# Use the containerized build explicitly
- make build-devserver-ctr
- echo -en 'travis_fold:end:build-devserver\\r'
- echo 'Building Developer JMS test image...' && echo -en 'travis_fold:start:build-devjmstest\\r'
- make build-devjmstest

View File

@@ -1,7 +1,29 @@
# Change log
## 9.1.3.0 (2019-07-19)
* Updated to MQ version 9.1.3.0
* Allow generation of TLS certificate with given hostname
* Fixes for the following issues:
* `MQ_EPHEMERAL_PREFIX` UNIX sockets fix
* Fix Makefile for Windows
* Use -a option on crtmqdir
* Remove check for certificate environment variable
## 9.1.2.0-UBI (2019-06-21)
**Breaking changes**:
* UID of the mqm user is now 888. You need to run the container with an entrypoint of `runmqserver -i` under the root user to update any existing files.
* MQSC files supplied will be verified before being run. Files containing invalid MQSC will cause the container to fail to start
**Other changes**:
* Security fixes
* Web console added to production image
* Container built on RedHat host
## 9.1.2.0 (2019-03-21)
* Updated to MQ version 9.1.2.0
* Now runs using the "mqm" user instead of root. See new [security doc](https://github.com/ibm-messaging/mq-container/blob/master/docs/security.md)
* New [IGNSTATE](https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_9.1.0/com.ibm.mq.pro.doc/q132310_.htm#q132310___ignstateparm) parameter used in default developer config
* Termination log moved from `/dev/termination-log` to `/run/termination-log`, to make permissions easier to handle

View File

@@ -12,24 +12,37 @@
# See the License for the specific language governing permissions and
# limitations under the License.
ARG BASE_IMAGE=ubuntu:16.04
ARG BUILDER_IMAGE=mq-golang-sdk:9.1.2.0-x86_64-ubuntu-16.04
ARG BASE_IMAGE=registry.access.redhat.com/ubi7/ubi-minimal
ARG BASE_TAG=7.6-237
###############################################################################
# Build stage to build Go code
###############################################################################
FROM $BUILDER_IMAGE as builder
WORKDIR /go/src/github.com/ibm-messaging/mq-container/
FROM registry.access.redhat.com/devtools/go-toolset-7-rhel7 as builder
# FROM docker.io/centos/go-toolset-7-centos7 as builder
# The URL to download the MQ installer from in tar.gz format
# This assumes an archive containing the MQ RPM install packages
ARG MQ_URL="https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqadv/mqadv_dev912_linux_x86-64.tar.gz"
ARG IMAGE_REVISION="Not specified"
ARG IMAGE_SOURCE="Not specified"
ARG IMAGE_TAG="Not specified"
ARG MQM_UID=888
USER 0
COPY install-mq.sh /usr/local/bin/
RUN chmod a+x /usr/local/bin/install-mq.sh \
&& sleep 1 \
&& MQ_PACKAGES="MQSeriesRuntime-*.rpm MQSeriesSDK-*.rpm MQSeriesSamples*.rpm" install-mq.sh $MQM_UID
WORKDIR /opt/app-root/src/go/src/github.com/ibm-messaging/mq-container/
COPY cmd/ ./cmd
COPY internal/ ./internal
COPY vendor/ ./vendor
ENV PATH="${PATH}:/opt/rh/go-toolset-7/root/usr/bin" \
CGO_CFLAGS="-I/opt/mqm/inc/" \
CGO_LDFLAGS_ALLOW="-Wl,-rpath.*"
RUN go build -ldflags "-X \"main.ImageCreated=$(date --iso-8601=seconds)\" -X \"main.ImageRevision=$IMAGE_REVISION\" -X \"main.ImageSource=$IMAGE_SOURCE\" -X \"main.ImageTag=$IMAGE_TAG\"" ./cmd/runmqserver/
RUN go build ./cmd/chkmqready/
RUN go build ./cmd/chkmqhealthy/
# Run all unit tests
RUN go build ./cmd/runmqdevserver/
RUN go test -v ./cmd/runmqdevserver/...
RUN go test -v ./cmd/runmqserver/
RUN go test -v ./cmd/chkmqready/
RUN go test -v ./cmd/chkmqhealthy/
@@ -39,46 +52,101 @@ RUN go vet ./cmd/... ./internal/...
###############################################################################
# Main build stage, to build MQ image
###############################################################################
FROM $BASE_IMAGE
# The URL to download the MQ installer from in tar.gz format
# This assumes an archive containing the MQ Debian (.deb) install packages
ARG MQ_URL
FROM $BASE_IMAGE:$BASE_TAG AS mq-server
# The MQ packages to install - see install-mq.sh for default value
ARG MQ_PACKAGES
# The UID to use for the "mqm" user
ARG MQM_UID=999
ARG MQ_URL="https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqadv/mqadv_dev912_linux_x86-64.tar.gz"
ARG MQ_PACKAGES="MQSeriesRuntime-*.rpm MQSeriesServer-*.rpm MQSeriesJava*.rpm MQSeriesJRE*.rpm MQSeriesGSKit*.rpm MQSeriesMsg*.rpm MQSeriesSamples*.rpm MQSeriesWeb*.rpm MQSeriesAMS-*.rpm"
#ARG MQ_PACKAGES="ibmmq-server ibmmq-java ibmmq-jre ibmmq-gskit ibmmq-msg-.* ibmmq-samples ibmmq-web ibmmq-ams"
ARG MQM_UID=888
ARG BASE_IMAGE
ARG BASE_TAG
LABEL summary="IBM MQ Advanced Server"
LABEL description="Simplify, accelerate and facilitate the reliable exchange of data with a security-rich messaging solution — trusted by the worlds most successful enterprises"
LABEL vendor="IBM"
LABEL distribution-scope="private"
LABEL authoritative-source-url="https://www.ibm.com/software/passportadvantage/"
LABEL url="https://www.ibm.com/products/mq/advanced"
LABEL io.openshift.tags="mq messaging"
LABEL io.k8s.display-name="IBM MQ Advanced Server"
LABEL io.k8s.description="Simplify, accelerate and facilitate the reliable exchange of data with a security-rich messaging solution — trusted by the worlds most successful enterprises"
LABEL base-image=$BASE_IMAGE
LABEL base-image-release=$BASE_TAG
COPY install-mq.sh /usr/local/bin/
COPY install-mq-server-prereqs.sh /usr/local/bin/
# Install MQ. To avoid a "text file busy" error here, we sleep before installing.
RUN chmod u+x /usr/local/bin/install-mq.sh \
RUN env && chmod u+x /usr/local/bin/install-*.sh \
&& sleep 1 \
&& install-mq-server-prereqs.sh $MQM_UID \
&& install-mq.sh $MQM_UID
# Create a directory for runtime data from runmqserver
RUN mkdir -p /run/runmqserver \
&& chown mqm:mqm /run/runmqserver
COPY --from=builder /go/src/github.com/ibm-messaging/mq-container/runmqserver /usr/local/bin/
COPY --from=builder /go/src/github.com/ibm-messaging/mq-container/chkmq* /usr/local/bin/
COPY --from=builder /opt/app-root/src/go/src/github.com/ibm-messaging/mq-container/runmqserver /usr/local/bin/
COPY --from=builder /opt/app-root/src/go/src/github.com/ibm-messaging/mq-container/chkmq* /usr/local/bin/
COPY NOTICES.txt /opt/mqm/licenses/notices-container.txt
# Copy web XML files
COPY web /etc/mqm/web
COPY etc/mqm/*.tpl /etc/mqm/
RUN chmod ug+x /usr/local/bin/runmqserver \
&& chown mqm:mqm /usr/local/bin/*mq* \
&& chmod ug+xs /usr/local/bin/chkmq* \
&& chown -R mqm:mqm /etc/mqm/* \
&& install --directory --mode 0775 --owner mqm --group root /run/runmqserver \
&& touch /run/termination-log \
&& chown mqm:root /run/termination-log \
&& chmod 0660 /run/termination-log
# Always use port 1414 for MQ & 9157 for the metrics
EXPOSE 1414 9157
EXPOSE 1414 9157 9443
ENV LANG=en_US.UTF-8 AMQ_DIAGNOSTIC_MSG_SEVERITY=1 AMQ_ADDITIONAL_JSON_LOG=1 LOG_FORMAT=basic
USER $MQM_UID
ENTRYPOINT ["runmqserver"]
###############################################################################
# Add default developer config
###############################################################################
FROM mq-server AS mq-dev-server
ARG MQM_UID=888
ARG BASE_IMAGE
ARG BASE_TAG
# Enable MQ developer default configuration
ENV MQ_DEV=true
# Default administrator password
ENV MQ_ADMIN_PASSWORD=passw0rd
LABEL summary="IBM MQ Advanced for Developers Server"
LABEL description="Simplify, accelerate and facilitate the reliable exchange of data with a security-rich messaging solution — trusted by the worlds most successful enterprises"
LABEL vendor="IBM"
LABEL distribution-scope="private"
LABEL authoritative-source-url="https://www.ibm.com/software/passportadvantage/"
LABEL url="https://www.ibm.com/products/mq/advanced"
LABEL io.openshift.tags="mq messaging"
LABEL io.k8s.display-name="IBM MQ Advanced for Developers Server"
LABEL io.k8s.description="Simplify, accelerate and facilitate the reliable exchange of data with a security-rich messaging solution — trusted by the worlds most successful enterprises"
LABEL base-image=$BASE_IMAGE
LABEL base-image-release=$BASE_TAG
USER 0
COPY incubating/mqadvanced-server-dev/install-extra-packages.sh /usr/local/bin/
RUN chmod u+x /usr/local/bin/install-extra-packages.sh \
&& sleep 1 \
&& install-extra-packages.sh
# WARNING: This is what allows the mqm user to change the password of any other user
# It's used by runmqdevserver to change the admin/app passwords.
RUN echo "mqm ALL = NOPASSWD: /usr/sbin/chpasswd" > /etc/sudoers.d/mq-dev-config
## Add admin and app users, and set a default password for admin
RUN useradd admin -G mqm \
&& groupadd mqclient \
&& useradd app -G mqclient \
&& echo admin:$MQ_ADMIN_PASSWORD | chpasswd
# Create a directory for runtime data from runmqserver
RUN mkdir -p /run/runmqdevserver \
&& chown mqm:mqm /run/runmqdevserver
COPY --from=builder /opt/app-root/src/go/src/github.com/ibm-messaging/mq-container/runmqdevserver /usr/local/bin/
# Copy template files
COPY incubating/mqadvanced-server-dev/*.tpl /etc/mqm/
# Copy web XML files for default developer configuration
COPY incubating/mqadvanced-server-dev/web /etc/mqm/web
RUN chown -R mqm:mqm /etc/mqm/* \
&& chmod +x /usr/local/bin/runmq* \
&& install --directory --mode 0775 --owner mqm --group root /run/runmqdevserver
ENV MQ_ENABLE_EMBEDDED_WEB_SERVER=1
USER $MQM_UID
ENTRYPOINT ["runmqdevserver"]

347
Makefile
View File

@@ -1,4 +1,4 @@
# © Copyright IBM Corporation 2018
# © Copyright IBM Corporation 2017, 2019
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -13,9 +13,63 @@
# limitations under the License.
###############################################################################
# Variables
# Conditional variables - you can override the values of these variables from
# the command line
###############################################################################
# MQ_VERSION is the fully qualified MQ version number to build
MQ_VERSION ?= 9.1.3.0
# RELEASE shows what release of the container code has been built
RELEASE ?= 2
# MQ_ARCHIVE is the name of the file, under the downloads directory, from which MQ Advanced can
# be installed. The default value is derived from MQ_VERSION, BASE_IMAGE and architecture
# Does not apply to MQ Advanced for Developers.
MQ_ARCHIVE ?= IBM_MQ_$(MQ_VERSION_VRM)_$(MQ_ARCHIVE_TYPE)_$(MQ_ARCHIVE_ARCH).tar.gz
# MQ_ARCHIVE_DEV is the name of the file, under the downloads directory, from which MQ Advanced
# for Developers can be installed
MQ_ARCHIVE_DEV ?= $(MQ_ARCHIVE_DEV_$(MQ_VERSION))
# MQ_SDK_ARCHIVE specifies the archive to use for building the golang programs. Defaults vary on developer or advanced.
MQ_SDK_ARCHIVE ?= $(MQ_ARCHIVE_DEV_$(MQ_VERSION))
# Options to `go test` for the Docker tests
TEST_OPTS_DOCKER ?=
# MQ_IMAGE_ADVANCEDSERVER is the name of the built MQ Advanced image
MQ_IMAGE_ADVANCEDSERVER ?=mqadvanced-server
# MQ_IMAGE_DEVSERVER is the name of the built MQ Advanced for Developers image
MQ_IMAGE_DEVSERVER ?=mqadvanced-server-dev
# MQ_TAG is the tag of the built MQ Advanced image & MQ Advanced for Developers image
MQ_TAG ?=$(MQ_VERSION)-$(ARCH)
# DOCKER is the Docker command to run. Defaults to "podman" if it's available, otherwise "docker"
DOCKER ?= $(shell type -p podman || echo docker)
# MQ_PACKAGES specifies the MQ packages (.deb or .rpm) to install. Defaults vary on base image.
MQ_PACKAGES ?=MQSeriesRuntime-*.rpm MQSeriesServer-*.rpm MQSeriesJava*.rpm MQSeriesJRE*.rpm MQSeriesGSKit*.rpm MQSeriesMsg*.rpm MQSeriesSamples*.rpm MQSeriesWeb*.rpm MQSeriesAMS-*.rpm
# MQM_UID is the UID to use for the "mqm" user
MQM_UID ?= 888
###############################################################################
# Other variables
###############################################################################
GO_PKG_DIRS = ./cmd ./internal ./test
MQ_ARCHIVE_TYPE=LINUX
MQ_ARCHIVE_DEV_PLATFORM=linux
# ARCH is the platform architecture (e.g. amd64, ppc64le or s390x)
ARCH=$(if $(findstring x86_64,$(shell uname -m)),amd64,$(shell uname -m))
# BUILD_SERVER_CONTAINER is the name of the web server container used at build time
BUILD_SERVER_CONTAINER=build-server
# NUM_CPU is the number of CPUs available to Docker. Used to control how many
# test run in parallel
NUM_CPU = $(or $(shell docker info --format "{{ .NCPU }}"),2)
# BASE_IMAGE_TAG is a normalized version of BASE_IMAGE, suitable for use in a Docker tag
BASE_IMAGE_TAG=$(lastword $(subst /, ,$(subst :,-,$(BASE_IMAGE))))
#BASE_IMAGE_TAG=$(subst /,-,$(subst :,-,$(BASE_IMAGE)))
MQ_IMAGE_DEVSERVER_BASE=mqadvanced-server-dev-base
# Docker image name to use for JMS tests
DEV_JMS_IMAGE=mq-dev-jms-test
# Variables for versioning
IMAGE_REVISION=$(shell git rev-parse HEAD)
IMAGE_SOURCE=$(shell git config --get remote.origin.url)
EMPTY:=
SPACE:= $(EMPTY) $(EMPTY)
# MQ_VERSION_VRM is MQ_VERSION with only the Version, Release and Modifier fields (no Fix field). e.g. 9.1.3 instead of 9.1.3.0
MQ_VERSION_VRM=$(subst $(SPACE),.,$(wordlist 1,3,$(subst .,$(SPACE),$(MQ_VERSION))))
# Set variable if running on a Red Hat Enterprise Linux host
ifneq ($(wildcard /etc/redhat-release),)
@@ -25,94 +79,246 @@ ifeq "$(findstring Red Hat,$(REDHAT_RELEASE))" "Red Hat"
endif
endif
ifneq (,$(findstring Microsoft,$(shell uname -r)))
DOWNLOADS_DIR=$(patsubst /mnt/c%,C:%,$(realpath ./downloads/))
else ifneq (,$(findstring Windows,$(shell echo ${OS})))
DOWNLOADS_DIR=$(shell pwd)/downloads/
else
DOWNLOADS_DIR=$(realpath ./downloads/)
endif
# Try to figure out which archive to use from the architecture
ifeq "$(ARCH)" "amd64"
MQ_ARCHIVE_ARCH=X86-64
MQ_DEV_ARCH=x86-64
else ifeq "$(ARCH)" "ppc64le"
MQ_ARCHIVE_ARCH=LE_POWER
MQ_DEV_ARCH=ppcle
else ifeq "$(ARCH)" "s390x"
MQ_ARCHIVE_ARCH=SYSTEM_Z
MQ_DEV_ARCH=s390x
endif
# Archive names for IBM MQ Advanced for Developers
MQ_ARCHIVE_DEV_9.1.0.0=mqadv_dev910_$(MQ_ARCHIVE_DEV_PLATFORM)_$(MQ_DEV_ARCH).tar.gz
MQ_ARCHIVE_DEV_9.1.1.0=mqadv_dev911_$(MQ_ARCHIVE_DEV_PLATFORM)_$(MQ_DEV_ARCH).tar.gz
MQ_ARCHIVE_DEV_9.1.2.0=mqadv_dev912_$(MQ_ARCHIVE_DEV_PLATFORM)_$(MQ_DEV_ARCH).tar.gz
MQ_ARCHIVE_DEV_9.1.3.0=mqadv_dev913_$(MQ_ARCHIVE_DEV_PLATFORM)_$(MQ_DEV_ARCH).tar.gz
###############################################################################
# Build targets
###############################################################################
.PHONY: vars
vars:
@echo $(MQ_ARCHIVE_ARCH)
@echo $(MQ_ARCHIVE_TYPE)
@echo $(MQ_ARCHIVE)
# Targets default to a RHEL image on a RHEL host, or an Ubuntu image everywhere else
.PHONY: default
default: build-devserver test
.PHONY: build-devserver
ifdef RHEL_HOST
build-devserver: build-devserver-rhel
else
build-devserver: build-devserver-ubuntu
endif
# Build all components (except incubating ones)
.PHONY: all
all: build-devserver build-advancedserver
.PHONY: test-all
test-all: build-devjmstest test-devserver test-advancedserver
.PHONY: devserver
devserver: build-devserver build-devjmstest test-devserver
# Build incubating components
.PHONY: incubating
incubating: build-explorer
downloads/$(MQ_ARCHIVE_DEV):
$(info $(SPACER)$(shell printf $(TITLE)"Downloading IBM MQ Advanced for Developers "$(MQ_VERSION)$(END)))
mkdir -p downloads
cd downloads; curl -LO https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqadv/$(MQ_ARCHIVE_DEV)
downloads/$(MQ_SDK_ARCHIVE):
$(info $(SPACER)$(shell printf $(TITLE)"Downloading IBM MQ Advanced for Developers "$(MQ_VERSION)$(END)))
mkdir -p downloads
cd downloads; curl -LO https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqadv/$(MQ_SDK_ARCHIVE)
.PHONY: downloads
downloads: downloads/$(MQ_ARCHIVE_DEV) downloads/$(MQ_SDK_ARCHIVE)
# Vendor Go dependencies for the Docker tests
test/docker/vendor:
cd test/docker && dep ensure -vendor-only
# Shortcut to just run the unit tests
.PHONY: test-unit
test-unit:
docker build --target builder --file Dockerfile-server .
.PHONY: test-advancedserver
test-advancedserver: test/docker/vendor
$(info $(SPACER)$(shell printf $(TITLE)"Test $(MQ_IMAGE_ADVANCEDSERVER):$(MQ_TAG) on $(shell docker --version)"$(END)))
docker inspect $(MQ_IMAGE_ADVANCEDSERVER):$(MQ_TAG)
cd test/docker && TEST_IMAGE=$(MQ_IMAGE_ADVANCEDSERVER):$(MQ_TAG) EXPECTED_LICENSE=Production go test -parallel $(NUM_CPU) $(TEST_OPTS_DOCKER)
.PHONY: build-devjmstest
build-devjmstest:
$(info $(SPACER)$(shell printf $(TITLE)"Build JMS tests for developer config"$(END)))
cd test/messaging && docker build --tag $(DEV_JMS_IMAGE) .
.PHONY: test-devserver
test-devserver: test/docker/vendor
$(info $(SPACER)$(shell printf $(TITLE)"Test $(MQ_IMAGE_DEVSERVER):$(MQ_TAG) on $(shell docker --version)"$(END)))
docker inspect $(MQ_IMAGE_DEVSERVER):$(MQ_TAG)
cd test/docker && TEST_IMAGE=$(MQ_IMAGE_DEVSERVER):$(MQ_TAG) EXPECTED_LICENSE=Developer DEV_JMS_IMAGE=$(DEV_JMS_IMAGE) IBMJRE=true go test -parallel $(NUM_CPU) -tags mqdev $(TEST_OPTS_DOCKER)
.PHONY: coverage
coverage:
mkdir coverage
.PHONY: test-advancedserver-cover
test-advancedserver-cover: test/docker/vendor coverage
$(info $(SPACER)$(shell printf $(TITLE)"Test $(MQ_IMAGE_ADVANCEDSERVER):$(MQ_TAG) with code coverage on $(shell docker --version)"$(END)))
rm -f ./coverage/unit*.cov
# Run unit tests with coverage, for each package under 'internal'
go list -f '{{.Name}}' ./internal/... | xargs -I {} go test -cover -covermode count -coverprofile ./coverage/unit-{}.cov ./internal/{}
# ls -1 ./cmd | xargs -I {} go test -cover -covermode count -coverprofile ./coverage/unit-{}.cov ./cmd/{}/...
echo 'mode: count' > ./coverage/unit.cov
tail -q -n +2 ./coverage/unit-*.cov >> ./coverage/unit.cov
go tool cover -html=./coverage/unit.cov -o ./coverage/unit.html
rm -f ./test/docker/coverage/*.cov
rm -f ./coverage/docker.*
mkdir -p ./test/docker/coverage/
cd test/docker && TEST_IMAGE=$(MQ_IMAGE_ADVANCEDSERVER):$(MQ_TAG)-cover TEST_COVER=true go test $(TEST_OPTS_DOCKER)
echo 'mode: count' > ./coverage/docker.cov
tail -q -n +2 ./test/docker/coverage/*.cov >> ./coverage/docker.cov
go tool cover -html=./coverage/docker.cov -o ./coverage/docker.html
echo 'mode: count' > ./coverage/combined.cov
tail -q -n +2 ./coverage/unit.cov ./coverage/docker.cov >> ./coverage/combined.cov
go tool cover -html=./coverage/combined.cov -o ./coverage/combined.html
define build-mq
# Create a temporary network to use for the build
$(DOCKER) network create build
# Start a web server to host the MQ downloadable (tar.gz) file
$(DOCKER) run \
--rm \
--name $(BUILD_SERVER_CONTAINER) \
--network build \
--network-alias build \
--volume $(DOWNLOADS_DIR):/usr/share/nginx/html:ro \
--detach \
docker.io/nginx:alpine
# Build the new image
$(DOCKER) build \
--tag $1:$2 \
--file $3 \
--network build \
--build-arg MQ_URL=http://build:80/$4 \
--build-arg MQ_PACKAGES="$(MQ_PACKAGES)" \
--build-arg IMAGE_REVISION="$(IMAGE_REVISION)" \
--build-arg IMAGE_SOURCE="$(IMAGE_SOURCE)" \
--build-arg IMAGE_TAG="$1:$2" \
--build-arg MQM_UID=$(MQM_UID) \
--label version=$(MQ_VERSION) \
--label name=$1 \
--label build-date=$(shell date +%Y-%m-%dT%H:%M:%S%z) \
--label release="$(RELEASE)" \
--label architecture="$(ARCH)" \
--label run="docker run -d -e LICENSE=accept $1:$2" \
--label vcs-ref=$(IMAGE_REVISION) \
--label vcs-type=git \
--label vcs-url=$(IMAGE_SOURCE) \
--target $5 \
. ; $(DOCKER) kill $(BUILD_SERVER_CONTAINER) && $(DOCKER) network rm build
endef
define build-mq-ctr
buildah/mq-buildah $1 $2 \
--file /src/Dockerfile-server \
--build-arg MQ_URL="file:///src/downloads/$3" \
--build-arg MQ_PACKAGES="$(MQ_PACKAGES)" \
--build-arg IMAGE_REVISION="$(IMAGE_REVISION)" \
--build-arg IMAGE_SOURCE="$(IMAGE_SOURCE)" \
--build-arg IMAGE_TAG="$1:$2" \
--build-arg MQM_UID=$(MQM_UID) \
--label version=$(MQ_VERSION) \
--label name=$1 \
--label build-date=$(shell date +%Y-%m-%dT%H:%M:%S%z) \
--label release="$(RELEASE)" \
--label architecture="$(ARCH)" \
--label run="docker run -d -e LICENSE=accept $1:$2" \
--label vcs-ref=$(IMAGE_REVISION) \
--label vcs-type=git \
--label vcs-url=$(IMAGE_SOURCE) \
--target $4
endef
DOCKER_SERVER_VERSION=$(shell docker version --format "{{ .Server.Version }}")
DOCKER_CLIENT_VERSION=$(shell docker version --format "{{ .Client.Version }}")
.PHONY: docker-version
docker-version:
@test "$(word 1,$(subst ., ,$(DOCKER_CLIENT_VERSION)))" -ge "17" || ("$(word 1,$(subst ., ,$(DOCKER_CLIENT_VERSION)))" -eq "17" && "$(word 2,$(subst ., ,$(DOCKER_CLIENT_VERSION)))" -ge "05") || (echo "Error: Docker client 17.05 or greater is required" && exit 1)
@test "$(word 1,$(subst ., ,$(DOCKER_SERVER_VERSION)))" -ge "17" || ("$(word 1,$(subst ., ,$(DOCKER_SERVER_VERSION)))" -eq "17" && "$(word 2,$(subst ., ,$(DOCKER_CLIENT_VERSION)))" -ge "05") || (echo "Error: Docker server 17.05 or greater is required" && exit 1)
.PHONY: build-advancedserver
ifdef RHEL_HOST
build-advancedserver: build-advancedserver-rhel
# Build using Buildah inside a container on RHEL hosts
build-advancedserver: build-advancedserver-ctr
else
build-advancedserver: build-advancedserver-ubuntu
build-advancedserver: build-advancedserver-host
endif
.PHONY: build-advancedserver-host
build-advancedserver-host: downloads/$(MQ_ARCHIVE) docker-version
$(info $(SPACER)$(shell printf $(TITLE)"Build $(MQ_IMAGE_ADVANCEDSERVER):$(MQ_TAG)"$(END)))
$(call build-mq,$(MQ_IMAGE_ADVANCEDSERVER),$(MQ_TAG),Dockerfile-server,$(MQ_ARCHIVE),mq-server)
.PHONY: test-devserver
.PHONY: build-advancedserver-ctr
build-advancedserver-ctr: downloads/$(MQ_ARCHIVE)
$(info $(shell printf $(TITLE)"Build $(MQ_IMAGE_ADVANCEDSERVER):$(MQ_TAG) in a container"$(END)))
$(call build-mq-ctr,$(MQ_IMAGE_ADVANCEDSERVER),$(MQ_TAG),$(MQ_ARCHIVE),mq-server)
.PHONY: build-devserver
ifdef RHEL_HOST
test-devserver: test-devserver-rhel
# Build using Buildah inside a container on RHEL hosts
build-devserver: build-devserver-ctr
else
test-devserver: test-devserver-ubuntu
build-devserver: build-devserver-host
endif
.PHONY: test-advancedserver
ifdef RHEL_HOST
test-advancedserver: test-advancedserver-rhel
else
test-advancedserver: test-advancedserver-ubuntu
endif
.PHONY: build-devserver-host
build-devserver-host: downloads/$(MQ_ARCHIVE_DEV) docker-version
$(info $(shell printf $(TITLE)"Build $(MQ_IMAGE_DEVSERVER):$(MQ_TAG)"$(END)))
$(call build-mq,$(MQ_IMAGE_DEVSERVER),$(MQ_TAG),Dockerfile-server,$(MQ_ARCHIVE_DEV),mq-dev-server)
.PHONY: build-devjmstest
ifdef RHEL_HOST
build-devjmstest: build-devjmstest-rhel
else
build-devjmstest: build-devjmstest-ubuntu
endif
.PHONY: build-devserver-ctr
build-devserver-ctr: downloads/$(MQ_ARCHIVE_DEV)
$(info $(shell printf $(TITLE)"Build $(MQ_IMAGE_DEVSERVER):$(MQ_TAG) in a container"$(END)))
$(call build-mq-ctr,$(MQ_IMAGE_DEVSERVER),$(MQ_TAG),$(MQ_ARCHIVE_DEV),mq-dev-server)
# UBUNTU building targets
.PHONY: build-devserver-ubuntu
build-devserver-ubuntu:
$(MAKE) -f Makefile-UBUNTU build-devserver
.PHONY: build-advancedserver-cover
build-advancedserver-cover: docker-version
$(DOCKER) build --build-arg BASE_IMAGE=$(MQ_IMAGE_ADVANCEDSERVER):$(MQ_TAG) -t $(MQ_IMAGE_ADVANCEDSERVER):$(MQ_TAG)-cover -f Dockerfile-server.cover .
.PHONY: test-devserver-ubuntu
test-devserver-ubuntu:
$(MAKE) -f Makefile-UBUNTU test-devserver
.PHONY: build-explorer
build-explorer: downloads/$(MQ_ARCHIVE_DEV)
$(call build-mq,mq-explorer,latest-$(ARCH),incubating/mq-explorer/Dockerfile,$(MQ_ARCHIVE_DEV),mq-explorer)
.PHONY: build-devjmstest-ubuntu
$(MAKE) -f Makefile-UBUNTU build-devjmstest
.PHONY: build-sdk
build-sdk: downloads/$(MQ_ARCHIVE_DEV)
$(info $(shell printf $(TITLE)"Build $(MQ_IMAGE_SDK)"$(END)))
$(call build-mq,mq-sdk,$(MQ_TAG),incubating/mq-sdk/Dockerfile,$(MQ_SDK_ARCHIVE),mq-sdk)
.PHONY: build-advancedserver-ubuntu
build-advancedserver-ubuntu:
$(MAKE) -f Makefile-UBUNTU build-advancedserver
.PHONY: debug-vars
debug-vars:
@echo MQ_VERSION=$(MQ_VERSION)
@echo MQ_VERSION_VRM=$(MQ_VERSION_VRM)
@echo MQ_ARCHIVE=$(MQ_ARCHIVE)
@echo MQ_IMAGE_DEVSERVER=$(MQ_IMAGE_DEVSERVER)
@echo MQ_IMAGE_ADVANCEDSERVER=$(MQ_IMAGE_ADVANCEDSERVER)
.PHONY: test-advancedserver-ubuntu
test-advancedserver-ubuntu:
$(MAKE) -f Makefile-UBUNTU test-advancedserver
include formatting.mk
.PHONY: build-devjmstest-ubuntu
build-devjmstest-ubuntu:
$(MAKE) -f Makefile-UBUNTU build-devjmstest
# RHEL building targets
.PHONY: build-devserver-rhel
build-devserver-rhel:
$(MAKE) -f Makefile-RHEL build-devserver
.PHONY: test-devserver-rhel
test-devserver-rhel:
$(MAKE) -f Makefile-RHEL test-devserver
.PHONY: build-advancedserver-rhel
build-advancedserver-rhel:
$(MAKE) -f Makefile-RHEL build-advancedserver
.PHONY: test-advancedserver-rhel
test-advancedserver-rhel:
$(MAKE) -f Makefile-RHEL test-advancedserver
.PHONY: build-devjmstest-rhel
build-devjmstest-rhel:
$(MAKE) -f Makefile-RHEL build-devjmstest
# Common targets
.PHONY: clean
clean:
rm -rf ./coverage
@@ -142,7 +348,7 @@ lint: $(addsuffix /$(wildcard *.go), $(GO_PKG_DIRS))
golint -set_exit_status $(sort $(dir $(wildcard $(addsuffix /*/*.go, $(GO_PKG_DIRS)))))
.PHONY: gosec
gosec: $(info $(SPACER)$(shell printf "Running gosec test"$(END)))
gosec: $(info $(SPACER)$(shell printf "Running gosec test"$(END)))
@gosec -fmt=json -out=gosec_results.json cmd/... internal/... 2> /dev/null ;\
cat "gosec_results.json" ;\
cat gosec_results.json | grep HIGH | grep severity > /dev/null ;\
@@ -167,9 +373,4 @@ gosec: $(info $(SPACER)$(shell printf "Running gosec test"$(END)))
printf "\ngosec found no LOW severity issues\n" ;\
fi ;\
.PHONY: unknownos
unknownos:
$(info $(SPACER)$(shell printf "ERROR: Unknown OS ("$(BASE_OS)") please run specific make targets"$(END)))
exit 1
include formatting.mk

View File

@@ -1,204 +0,0 @@
# © Copyright IBM Corporation 2018, 2019
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
###############################################################################
# Conditional variables - you can override the values of these variables from
# the command line
###############################################################################
# BASE_IMAGE is the base image to use for MQ, for example "ubuntu" or "rhel"
BASE_IMAGE ?= rhel
# MQ_VERSION is the fully qualified MQ version number to build
MQ_VERSION ?= 9.1.2.0
# MQ_ARCHIVE is the name of the file, under the downloads directory, from which MQ Advanced can
# be installed. The default value is derived from MQ_VERSION, BASE_IMAGE and architecture
# Does not apply to MQ Advanced for Developers.
MQ_ARCHIVE ?= IBM_MQ_$(MQ_VERSION_VRM)_LINUX_$(MQ_ARCHIVE_ARCH).tar.gz
# MQ_ARCHIVE_DEV is the name of the file, under the downloads directory, from which MQ Advanced
# for Developers can be installed
MQ_ARCHIVE_DEV ?= $(MQ_ARCHIVE_DEV_$(MQ_VERSION))
# MQ_SDK_ARCHIVE specifies the archive to use for the MQ redistributable client, which is used for building the golang programs.
MQ_SDK_ARCHIVE ?= 9.1.2.0-IBM-MQC-Redist-LinuxX64.tar.gz
# Options to `go test` for the Docker tests
TEST_OPTS_DOCKER ?=
# MQ_IMAGE_ADVANCEDSERVER is the name and tag of the built MQ Advanced image
MQ_IMAGE_ADVANCEDSERVER ?=mqadvanced-server:$(MQ_VERSION)-RHEL-$(ARCH)
# MQ_IMAGE_DEVSERVER is the name and tag of the built MQ Advanced for Developers image
MQ_IMAGE_DEVSERVER ?=mqadvanced-server-dev:$(MQ_VERSION)-RHEL-$(ARCH)
# MQ_IMAGE_SDK is the name and tag of the built MQ Advanced for Developers SDK image
MQ_IMAGE_SDK ?=mq-sdk:$(MQ_VERSION)-$(ARCH)-$(BASE_IMAGE_TAG)
# MQ_IMAGE_GOLANG_SDK is the name and tag of the built MQ Advanced for Developers SDK image, plus Go tools
MQ_IMAGE_GOLANG_SDK ?=mq-golang-sdk:$(MQ_VERSION)-$(ARCH)-$(BASE_IMAGE_TAG)
# MQ_PACKAGES specifies the MQ packages to install. Defaults vary on base image.
MQ_PACKAGES ?= MQSeriesRuntime-*.rpm MQSeriesServer-*.rpm MQSeriesJava*.rpm MQSeriesJRE*.rpm MQSeriesGSKit*.rpm MQSeriesMsg*.rpm MQSeriesSamples*.rpm MQSeriesAMS-*.rpm
###############################################################################
# Other variables
###############################################################################
# ARCH is the platform architecture (e.g. x86_64, ppc64le or s390x)
ARCH = $(shell uname -m)
# BASE_IMAGE_TAG is a normalized version of BASE_IMAGE, suitable for use in a Docker tag
BASE_IMAGE_TAG=$(subst /,-,$(subst :,-,$(BASE_IMAGE)))
MQ_IMAGE_DEVSERVER_BASE=mqadvanced-server-dev-base:$(MQ_VERSION)-$(ARCH)-$(BASE_IMAGE_TAG)
# Docker image name to use for JMS tests
DEV_JMS_IMAGE=mq-dev-jms-test:latest
# Variables for versioning
IMAGE_REVISION=$(shell git rev-parse HEAD)
IMAGE_SOURCE=$(shell git config --get remote.origin.url)
MQDEV=
EMPTY:=
SPACE:= $(EMPTY) $(EMPTY)
# MQ_VERSION_VRM is MQ_VERSION with only the Version, Release and Modifier fields (no Fix field). e.g. 9.1.2 instead of 9.1.2.0
MQ_VERSION_VRM=$(subst $(SPACE),.,$(wordlist 1,3,$(subst .,$(SPACE),$(MQ_VERSION))))
ifneq (,$(findstring Microsoft,$(shell uname -r)))
DOWNLOADS_DIR=$(patsubst /mnt/c%,C:%,$(realpath ./downloads/))
else
DOWNLOADS_DIR=$(realpath ./downloads/)
endif
# Try to figure out which archive to use from the architecture
ifeq "$(ARCH)" "x86_64"
MQ_ARCHIVE_ARCH=X86-64
MQ_DEV_ARCH=x86-64
else ifeq "$(ARCH)" "ppc64le"
MQ_ARCHIVE_ARCH=LE_POWER
MQ_DEV_ARCH=ppcle
else ifeq "$(ARCH)" "s390x"
MQ_ARCHIVE_ARCH=SYSTEM_Z
MQ_DEV_ARCH=s390x
endif
# Archive names for IBM MQ Advanced for Developers
MQ_ARCHIVE_DEV_9.1.0.0=mqadv_dev910_linux_$(MQ_DEV_ARCH).tar.gz
MQ_ARCHIVE_DEV_9.1.1.0=mqadv_dev911_linux_$(MQ_DEV_ARCH).tar.gz
MQ_ARCHIVE_DEV_9.1.2.0=mqadv_dev912_linux_$(MQ_DEV_ARCH).tar.gz
###############################################################################
# Build targets
###############################################################################
.PHONY: vars
vars:
#ifeq "$(findstring ubuntu,$(BASE_IMAGE))","ubuntu"
@echo $(MQ_ARCHIVE_ARCH)
@echo $(MQ_ARCHIVE_TYPE)
@echo $(MQ_ARCHIVE)
.PHONY: default
default: build-devserver test-devserver
# Build all components (except incubating ones)
.PHONY: all
all: build-devserver build-advancedserver
.PHONY: test-all
test-all: build-devjmstest test-devserver test-advancedserver
.PHONY: devserver
devserver: build-devserver build-devjmstest test-devserver
# Build incubating components
.PHONY: incubating
incubating: build-explorer
downloads/$(MQ_ARCHIVE_DEV):
$(info $(SPACER)$(shell printf $(TITLE)"Downloading IBM MQ Advanced for Developers "$(MQ_VERSION)$(END)))
mkdir -p downloads
cd downloads; curl -LO https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqadv/$(MQ_ARCHIVE_DEV)
downloads/$(MQ_SDK_ARCHIVE):
$(info $(SPACER)$(shell printf $(TITLE)"Downloading IBM MQ Advanced redistributable client "$(MQ_VERSION)$(END)))
mkdir -p downloads
cd downloads; curl -LO https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqdev/redist/$(MQ_SDK_ARCHIVE)
.PHONY: downloads
downloads: downloads/$(MQ_ARCHIVE_DEV) downloads/$(MQ_SDK_ARCHIVE)
# Vendor Go dependencies for the Docker tests
test/docker/vendor:
cd test/docker && dep ensure -vendor-only
.PHONY: check-prereqs
check-prereqs:
$(info $(SPACER)$(shell printf $(TITLE)"Checking for prereqs"$(END)))
which buildah || (echo "Missing required program buildah" && exit 1)
which podman || (echo "Missing required program podman" && exit 1)
yum list | grep yum-utils || (echo "Missing required package yum-utils" && exit 1)
.PHONY: check-test-prereqs
check-test-prereqs:
$(info $(SPACER)$(shell printf $(TITLE)"Checking for prereqs"$(END)))
which buildah || (echo "Missing required program buildah" && exit 1)
which docker || (echo "Missing required program docker" && exit 1)
.PHONY: test-advancedserver
test-advancedserver: check-test-prereqs test/docker/vendor
$(info $(SPACER)$(shell printf $(TITLE)"Test $(MQ_IMAGE_ADVANCEDSERVER) on $(shell docker --version)"$(END)))
sudo buildah push $(MQ_IMAGE_ADVANCEDSERVER) docker-daemon:$(MQ_IMAGE_ADVANCEDSERVER)
docker tag docker.io/$(MQ_IMAGE_ADVANCEDSERVER) $(MQ_IMAGE_ADVANCEDSERVER)
cd test/docker && TEST_IMAGE=$(MQ_IMAGE_ADVANCEDSERVER) EXPECTED_LICENSE=Production go test $(TEST_OPTS_DOCKER)
.PHONY: test-devserver
test-devserver: check-test-prereqs test/docker/vendor
$(info $(SPACER)$(shell printf $(TITLE)"Test $(MQ_IMAGE_DEVSERVER) on $(shell docker --version)"$(END)))
sudo buildah push $(MQ_IMAGE_DEVSERVER) docker-daemon:$(MQ_IMAGE_DEVSERVER)
docker tag docker.io/$(MQ_IMAGE_DEVSERVER) $(MQ_IMAGE_DEVSERVER)
cd test/docker && TEST_IMAGE=$(MQ_IMAGE_DEVSERVER) EXPECTED_LICENSE=Developer DEV_JMS_IMAGE=$(DEV_JMS_IMAGE) go test -tags mqdev $(TEST_OPTS_DOCKER)
.PHONY: build-advancedserver
build-advancedserver: check-prereqs downloads/$(MQ_ARCHIVE) build-go-programs
$(info $(SPACER)$(shell printf $(TITLE)"Build $(MQ_IMAGE_ADVANCEDSERVER)"$(END)))
sudo mq-advanced-server-rhel/mq-buildah.sh "$(MQ_ARCHIVE)" "$(MQ_PACKAGES)" "$(MQ_IMAGE_ADVANCEDSERVER)" "$(MQ_VERSION)" "$(MQDEV)"
.PHONY: build-devserver
build-devserver: MQDEV=TRUE
build-devserver: MQ_PACKAGES=MQSeriesRuntime-*.rpm MQSeriesServer-*.rpm MQSeriesJava*.rpm MQSeriesJRE*.rpm MQSeriesGSKit*.rpm MQSeriesMsg*.rpm MQSeriesSamples*.rpm MQSeriesAMS-*.rpm MQSeriesWeb-*.rpm
build-devserver: check-prereqs downloads/$(MQ_ARCHIVE_DEV) build-go-programs
$(info $(SPACER)$(shell printf $(TITLE)"Build $(MQ_IMAGE_DEVSERVER)"$(END)))
sudo mq-advanced-server-rhel/mq-buildah.sh "$(MQ_ARCHIVE_DEV)" "$(MQ_PACKAGES)" "$(MQ_IMAGE_DEVSERVER_BASE)" "$(MQ_VERSION)" "$(MQDEV)"
sudo mq-advanced-server-rhel/mqdev-buildah.sh "$(MQ_IMAGE_DEVSERVER_BASE)" "$(MQ_IMAGE_DEVSERVER)" "$(MQ_VERSION)"
.PHONY: build-mqgolang-sdk
build-mqgolang-sdk: check-prereqs downloads/$(MQ_SDK_ARCHIVE)
$(info $(SPACER)$(shell printf $(TITLE)"Build mq-golang SDK"$(END)))
sudo mq-advanced-server-rhel/mq-golang-sdk-buildah.sh "$(MQ_SDK_ARCHIVE)" "$(MQ_IMAGE_GOLANG_SDK)"
.PHONY: build-go-programs
build-go-programs: check-prereqs downloads/$(MQ_SDK_ARCHIVE) build-mqgolang-sdk
$(info $(SPACER)$(shell printf $(TITLE)"Build go programs"$(END)))
IMAGE_REVISION=$(IMAGE_REVISION) IMAGE_SOURCE=$(IMAGE_SOURCE) sudo --preserve-env mq-advanced-server-rhel/go-buildah.sh "$(MQ_IMAGE_GOLANG_SDK)" "$(MQDEV)"
.PHONY: build-devjmstest
build-devjmstest: check-test-prereqs
$(info $(SPACER)$(shell printf $(TITLE)"Build JMS tests for developer config"$(END)))
cd test/messaging && sudo ./buildah.sh $(DEV_JMS_IMAGE)
sudo buildah push $(DEV_JMS_IMAGE) docker-daemon:$(DEV_JMS_IMAGE)
docker tag docker.io/$(DEV_JMS_IMAGE) $(DEV_JMS_IMAGE)
.PHONY: debug-vars
debug-vars:
@echo MQ_VERSION=$(MQ_VERSION)
@echo MQ_VERSION_VRM=$(MQ_VERSION_VRM)
@echo MQ_ARCHIVE=$(MQ_ARCHIVE)
@echo MQ_SDK_ARCHIVE=$(MQ_SDK_ARCHIVE)
@echo MQ_IMAGE_GOLANG_SDK=$(MQ_IMAGE_GOLANG_SDK)
@echo MQ_IMAGE_DEVSERVER_BASE=$(MQ_IMAGE_DEVSERVER_BASE)
@echo MQ_IMAGE_DEVSERVER=$(MQ_IMAGE_DEVSERVER)
@echo MQ_IMAGE_ADVANCEDSERVER=$(MQ_IMAGE_ADVANCEDSERVER)
include formatting.mk

View File

@@ -1,291 +0,0 @@
# © Copyright IBM Corporation 2017, 2019
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
###############################################################################
# Conditional variables - you can override the values of these variables from
# the command line
###############################################################################
# BASE_IMAGE is the base image to use for MQ, for example "ubuntu" or "rhel"
BASE_IMAGE ?= ubuntu:16.04
# MQ_VERSION is the fully qualified MQ version number to build
MQ_VERSION ?= 9.1.2.0
# MQ_ARCHIVE is the name of the file, under the downloads directory, from which MQ Advanced can
# be installed. The default value is derived from MQ_VERSION, BASE_IMAGE and architecture
# Does not apply to MQ Advanced for Developers.
MQ_ARCHIVE ?= IBM_MQ_$(MQ_VERSION_VRM)_$(MQ_ARCHIVE_TYPE)_$(MQ_ARCHIVE_ARCH).tar.gz
# MQ_ARCHIVE_DEV is the name of the file, under the downloads directory, from which MQ Advanced
# for Developers can be installed
MQ_ARCHIVE_DEV ?= $(MQ_ARCHIVE_DEV_$(MQ_VERSION))
# MQ_SDK_ARCHIVE specifies the archive to use for building the golang programs. Defaults vary on developer or advanced.
MQ_SDK_ARCHIVE ?= $(MQ_ARCHIVE_DEV_$(MQ_VERSION))
# Options to `go test` for the Docker tests
TEST_OPTS_DOCKER ?=
# MQ_IMAGE_ADVANCEDSERVER is the name and tag of the built MQ Advanced image
MQ_IMAGE_ADVANCEDSERVER ?=mqadvanced-server:$(MQ_VERSION)-$(ARCH)-$(BASE_IMAGE_TAG)
# MQ_IMAGE_DEVSERVER is the name and tag of the built MQ Advanced for Developers image
MQ_IMAGE_DEVSERVER ?=mqadvanced-server-dev:$(MQ_VERSION)-$(ARCH)-$(BASE_IMAGE_TAG)
# MQ_IMAGE_SDK is the name and tag of the built MQ Advanced for Developers SDK image
MQ_IMAGE_SDK ?=mq-sdk:$(MQ_VERSION)-$(ARCH)-$(BASE_IMAGE_TAG)
# MQ_IMAGE_GOLANG_SDK is the name and tag of the built MQ Advanced for Developers SDK image, plus Go tools
MQ_IMAGE_GOLANG_SDK ?=mq-golang-sdk:$(MQ_VERSION)-$(ARCH)-$(BASE_IMAGE_TAG)
# DOCKER is the Docker command to run
DOCKER ?= docker
# MQ_PACKAGES specifies the MQ packages (.deb or .rpm) to install. Defaults vary on base image.
MQ_PACKAGES ?=
###############################################################################
# Other variables
###############################################################################
# ARCH is the platform architecture (e.g. x86_64, ppc64le or s390x)
ARCH = $(shell uname -m)
# BUILD_SERVER_CONTAINER is the name of the web server container used at build time
BUILD_SERVER_CONTAINER=build-server
# NUM_CPU is the number of CPUs available to Docker. Used to control how many
# test run in parallel
NUM_CPU = $(or $(shell docker info --format "{{ .NCPU }}"),2)
# BASE_IMAGE_TAG is a normalized version of BASE_IMAGE, suitable for use in a Docker tag
BASE_IMAGE_TAG=$(subst /,-,$(subst :,-,$(BASE_IMAGE)))
MQ_IMAGE_DEVSERVER_BASE=mqadvanced-server-dev-base:$(MQ_VERSION)-$(ARCH)-$(BASE_IMAGE_TAG)
# Docker image name to use for JMS tests
DEV_JMS_IMAGE=mq-dev-jms-test
# Variables for versioning
IMAGE_REVISION=$(shell git rev-parse HEAD)
IMAGE_SOURCE=$(shell git config --get remote.origin.url)
EMPTY:=
SPACE:= $(EMPTY) $(EMPTY)
# MQ_VERSION_VRM is MQ_VERSION with only the Version, Release and Modifier fields (no Fix field). e.g. 9.1.2 instead of 9.1.2.0
MQ_VERSION_VRM=$(subst $(SPACE),.,$(wordlist 1,3,$(subst .,$(SPACE),$(MQ_VERSION))))
ifneq (,$(findstring Microsoft,$(shell uname -r)))
DOWNLOADS_DIR=$(patsubst /mnt/c%,C:%,$(realpath ./downloads/))
else
DOWNLOADS_DIR=$(realpath ./downloads/)
endif
# Try to figure out which archive to use from the BASE_IMAGE
ifeq "$(findstring ubuntu,$(BASE_IMAGE))" "ubuntu"
MQ_ARCHIVE_TYPE=UBUNTU
MQ_ARCHIVE_DEV_PLATFORM=ubuntu
MQM_UID=999
else
MQ_ARCHIVE_TYPE=LINUX
MQ_ARCHIVE_DEV_PLATFORM=linux
MQM_UID=888
endif
# Try to figure out which archive to use from the architecture
ifeq "$(ARCH)" "x86_64"
MQ_ARCHIVE_ARCH=X86-64
MQ_DEV_ARCH=x86-64
else ifeq "$(ARCH)" "ppc64le"
MQ_ARCHIVE_ARCH=LE_POWER
MQ_DEV_ARCH=ppcle
else ifeq "$(ARCH)" "s390x"
MQ_ARCHIVE_ARCH=SYSTEM_Z
MQ_DEV_ARCH=s390x
endif
# Archive names for IBM MQ Advanced for Developers
MQ_ARCHIVE_DEV_9.1.0.0=mqadv_dev910_$(MQ_ARCHIVE_DEV_PLATFORM)_$(MQ_DEV_ARCH).tar.gz
MQ_ARCHIVE_DEV_9.1.1.0=mqadv_dev911_$(MQ_ARCHIVE_DEV_PLATFORM)_$(MQ_DEV_ARCH).tar.gz
MQ_ARCHIVE_DEV_9.1.2.0=mqadv_dev912_$(MQ_ARCHIVE_DEV_PLATFORM)_$(MQ_DEV_ARCH).tar.gz
###############################################################################
# Build targets
###############################################################################
.PHONY: vars
vars:
#ifeq "$(findstring ubuntu,$(BASE_IMAGE))","ubuntu"
@echo $(MQ_ARCHIVE_ARCH)
@echo $(MQ_ARCHIVE_TYPE)
@echo $(MQ_ARCHIVE)
.PHONY: default
default: build-devserver test
# Build all components (except incubating ones)
.PHONY: all
all: build-devserver build-advancedserver
.PHONY: test-all
test-all: build-devjmstest test-devserver test-advancedserver
.PHONY: devserver
devserver: build-devserver build-devjmstest test-devserver
# Build incubating components
.PHONY: incubating
incubating: build-explorer
downloads/$(MQ_ARCHIVE_DEV):
$(info $(SPACER)$(shell printf $(TITLE)"Downloading IBM MQ Advanced for Developers "$(MQ_VERSION)$(END)))
mkdir -p downloads
cd downloads; curl -LO https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqadv/$(MQ_ARCHIVE_DEV)
downloads/$(MQ_SDK_ARCHIVE):
$(info $(SPACER)$(shell printf $(TITLE)"Downloading IBM MQ Advanced for Developers "$(MQ_VERSION)$(END)))
mkdir -p downloads
cd downloads; curl -LO https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqadv/$(MQ_SDK_ARCHIVE)
.PHONY: downloads
downloads: downloads/$(MQ_ARCHIVE_DEV) downloads/$(MQ_SDK_ARCHIVE)
# Vendor Go dependencies for the Docker tests
test/docker/vendor:
cd test/docker && dep ensure -vendor-only
# Shortcut to just run the unit tests
.PHONY: test-unit
test-unit:
docker build --target builder --file Dockerfile-server .
.PHONY: test-advancedserver
test-advancedserver: test/docker/vendor
$(info $(SPACER)$(shell printf $(TITLE)"Test $(MQ_IMAGE_ADVANCEDSERVER) on $(shell docker --version)"$(END)))
docker inspect $(MQ_IMAGE_ADVANCEDSERVER)
cd test/docker && TEST_IMAGE=$(MQ_IMAGE_ADVANCEDSERVER) EXPECTED_LICENSE=Production go test -parallel $(NUM_CPU) $(TEST_OPTS_DOCKER)
.PHONY: build-devjmstest
build-devjmstest:
$(info $(SPACER)$(shell printf $(TITLE)"Build JMS tests for developer config"$(END)))
cd test/messaging && docker build --tag $(DEV_JMS_IMAGE) .
.PHONY: test-devserver
test-devserver: test/docker/vendor
$(info $(SPACER)$(shell printf $(TITLE)"Test $(MQ_IMAGE_DEVSERVER) on $(shell docker --version)"$(END)))
docker inspect $(MQ_IMAGE_DEVSERVER)
cd test/docker && TEST_IMAGE=$(MQ_IMAGE_DEVSERVER) EXPECTED_LICENSE=Developer DEV_JMS_IMAGE=$(DEV_JMS_IMAGE) IBMJRE=true go test -parallel $(NUM_CPU) -tags mqdev $(TEST_OPTS_DOCKER)
coverage:
mkdir coverage
.PHONY: test-advancedserver-cover
test-advancedserver-cover: test/docker/vendor coverage
$(info $(SPACER)$(shell printf $(TITLE)"Test $(MQ_IMAGE_ADVANCEDSERVER) with code coverage on $(shell docker --version)"$(END)))
rm -f ./coverage/unit*.cov
# Run unit tests with coverage, for each package under 'internal'
go list -f '{{.Name}}' ./internal/... | xargs -I {} go test -cover -covermode count -coverprofile ./coverage/unit-{}.cov ./internal/{}
# ls -1 ./cmd | xargs -I {} go test -cover -covermode count -coverprofile ./coverage/unit-{}.cov ./cmd/{}/...
echo 'mode: count' > ./coverage/unit.cov
tail -q -n +2 ./coverage/unit-*.cov >> ./coverage/unit.cov
go tool cover -html=./coverage/unit.cov -o ./coverage/unit.html
rm -f ./test/docker/coverage/*.cov
rm -f ./coverage/docker.*
mkdir -p ./test/docker/coverage/
cd test/docker && TEST_IMAGE=$(MQ_IMAGE_ADVANCEDSERVER)-cover TEST_COVER=true go test $(TEST_OPTS_DOCKER)
echo 'mode: count' > ./coverage/docker.cov
tail -q -n +2 ./test/docker/coverage/*.cov >> ./coverage/docker.cov
go tool cover -html=./coverage/docker.cov -o ./coverage/docker.html
echo 'mode: count' > ./coverage/combined.cov
tail -q -n +2 ./coverage/unit.cov ./coverage/docker.cov >> ./coverage/combined.cov
go tool cover -html=./coverage/combined.cov -o ./coverage/combined.html
define docker-build-mq
# Create a temporary network to use for the build
$(DOCKER) network create build
# Start a web server to host the MQ downloadable (tar.gz) file
$(DOCKER) run \
--rm \
--name $(BUILD_SERVER_CONTAINER) \
--network build \
--network-alias build \
--volume $(DOWNLOADS_DIR):/usr/share/nginx/html:ro \
--detach \
nginx:alpine
# Build the new image
$(DOCKER) build \
--tag $1 \
--file $2 \
--network build \
--build-arg MQ_URL=http://build:80/$3 \
--build-arg BASE_IMAGE=$(BASE_IMAGE) \
--build-arg BUILDER_IMAGE=$(MQ_IMAGE_GOLANG_SDK) \
--build-arg IMAGE_REVISION="$(IMAGE_REVISION)" \
--build-arg IMAGE_SOURCE="$(IMAGE_SOURCE)" \
--build-arg IMAGE_TAG="$1" \
--build-arg MQM_UID=$(MQM_UID) \
--label IBM_PRODUCT_ID=$4 \
--label IBM_PRODUCT_NAME=$5 \
--label IBM_PRODUCT_VERSION=$6 \
--build-arg MQ_PACKAGES="$(MQ_PACKAGES)" \
. ; $(DOCKER) kill $(BUILD_SERVER_CONTAINER) && $(DOCKER) network rm build
endef
DOCKER_SERVER_VERSION=$(shell docker version --format "{{ .Server.Version }}")
DOCKER_CLIENT_VERSION=$(shell docker version --format "{{ .Client.Version }}")
.PHONY: docker-version
docker-version:
@test "$(word 1,$(subst ., ,$(DOCKER_CLIENT_VERSION)))" -ge "17" || ("$(word 1,$(subst ., ,$(DOCKER_CLIENT_VERSION)))" -eq "17" && "$(word 2,$(subst ., ,$(DOCKER_CLIENT_VERSION)))" -ge "05") || (echo "Error: Docker client 17.05 or greater is required" && exit 1)
@test "$(word 1,$(subst ., ,$(DOCKER_SERVER_VERSION)))" -ge "17" || ("$(word 1,$(subst ., ,$(DOCKER_SERVER_VERSION)))" -eq "17" && "$(word 2,$(subst ., ,$(DOCKER_CLIENT_VERSION)))" -ge "05") || (echo "Error: Docker server 17.05 or greater is required" && exit 1)
.PHONY: build-advancedserver
build-advancedserver: MQ_SDK_ARCHIVE=$(MQ_ARCHIVE)
build-advancedserver: downloads/$(MQ_ARCHIVE) docker-version build-golang-sdk-ex
$(info $(SPACER)$(shell printf $(TITLE)"Build $(MQ_IMAGE_ADVANCEDSERVER)"$(END)))
$(call docker-build-mq,$(MQ_IMAGE_ADVANCEDSERVER),Dockerfile-server,$(MQ_ARCHIVE),"4486e8c4cc9146fd9b3ce1f14a2dfc5b","IBM MQ Advanced",$(MQ_VERSION))
.PHONY: build-devserver
# Target-specific variable to add web server into devserver image
ifeq "$(findstring ubuntu,$(BASE_IMAGE))" "ubuntu"
build-devserver: MQ_PACKAGES=ibmmq-server ibmmq-java ibmmq-jre ibmmq-gskit ibmmq-msg-.* ibmmq-samples ibmmq-ams ibmmq-web
else
build-devserver: MQ_PACKAGES=MQSeriesRuntime-*.rpm MQSeriesServer-*.rpm MQSeriesJava*.rpm MQSeriesJRE*.rpm MQSeriesGSKit*.rpm MQSeriesMsg*.rpm MQSeriesSamples*.rpm MQSeriesAMS-*.rpm MQSeriesWeb-*.rpm
endif
build-devserver: MQ_SDK_ARCHIVE=$(MQ_ARCHIVE_DEV)
build-devserver: downloads/$(MQ_ARCHIVE_DEV) docker-version build-golang-sdk-ex
$(info $(shell printf $(TITLE)"Build $(MQ_IMAGE_DEVSERVER_BASE)"$(END)))
$(call docker-build-mq,$(MQ_IMAGE_DEVSERVER_BASE),Dockerfile-server,$(MQ_ARCHIVE_DEV),"98102d16795c4263ad9ca075190a2d4d","IBM MQ Advanced for Developers (Non-Warranted)",$(MQ_VERSION))
$(DOCKER) build --tag $(MQ_IMAGE_DEVSERVER) --build-arg IMAGE_SOURCE="$(IMAGE_SOURCE)" --build-arg IMAGE_REVISION="$(IMAGE_REVISION)" --build-arg IMAGE_TAG="$(MQ_IMAGE_DEVSERVER)" --build-arg BASE_IMAGE=$(MQ_IMAGE_DEVSERVER_BASE) --build-arg BUILDER_IMAGE=$(MQ_IMAGE_GOLANG_SDK) --build-arg MQM_UID=$(MQM_UID) --file incubating/mqadvanced-server-dev/Dockerfile .
.PHONY: build-advancedserver-cover
build-advancedserver-cover: docker-version
$(DOCKER) build --build-arg BASE_IMAGE=$(MQ_IMAGE_ADVANCEDSERVER) -t $(MQ_IMAGE_ADVANCEDSERVER)-cover -f Dockerfile-server.cover .
.PHONY: build-explorer
ifeq "$(findstring ubuntu,$(BASE_IMAGE))" "ubuntu"
build-explorer: MQ_PACKAGES=ibmmq-explorer
else
build-explorer: MQ_PACKAGES=MQSeriesRuntime*.rpm MQSeriesJRE*.rpm MQSeriesExplorer*.rpm
endif
build-explorer: downloads/$(MQ_ARCHIVE_DEV) docker-pull
$(call docker-build-mq,mq-explorer:latest-$(ARCH),incubating/mq-explorer/Dockerfile,$(MQ_ARCHIVE_DEV),"98102d16795c4263ad9ca075190a2d4d","IBM MQ Advanced for Developers (Non-Warranted)",$(MQ_VERSION))
.PHONY: build-sdk
build-sdk: downloads/$(MQ_SDK_ARCHIVE) build-sdk-ex
.PHONY: build-sdk-ex
ifeq "$(findstring ubuntu,$(BASE_IMAGE))" "ubuntu"
build-sdk-ex: MQ_PACKAGES=ibmmq-sdk ibmmq-samples build-essential
else
build-sdk-ex: MQ_PACKAGES=MQSeriesRuntime-*.rpm MQSeriesSDK-*.rpm MQSeriesSamples*.rpm
endif
build-sdk-ex: docker-version docker-pull
$(info $(shell printf $(TITLE)"Build $(MQ_IMAGE_SDK)"$(END)))
$(call docker-build-mq,$(MQ_IMAGE_SDK),incubating/mq-sdk/Dockerfile,$(MQ_SDK_ARCHIVE),"98102d16795c4263ad9ca075190a2d4d","IBM MQ Advanced for Developers SDK (Non-Warranted)",$(MQ_VERSION))
.PHONY: build-golang-sdk
build-golang-sdk: downloads/$(MQ_SDK_ARCHIVE) build-golang-sdk-ex
.PHONY: build-golang-sdk-ex
build-golang-sdk-ex: docker-version build-sdk-ex
$(info $(shell printf $(TITLE)"Build $(MQ_IMAGE_GOLANG_SDK)"$(END)))
@echo hello
$(DOCKER) build --build-arg BASE_IMAGE=$(MQ_IMAGE_SDK) -t $(MQ_IMAGE_GOLANG_SDK) -f incubating/mq-golang-sdk/Dockerfile .
@echo goodbye
.PHONY: docker-pull
docker-pull:
$(DOCKER) pull $(BASE_IMAGE)
include formatting.mk

View File

@@ -5,8 +5,6 @@
**Note**: The `master` branch may be in an *unstable or even broken state* during development.
To get a stable version, please use the correct [branch](https://github.com/ibm-messaging/mq-container/branches) for your MQ version, instead of the `master` branch.
<img src="https://raw.githubusercontent.com/IBM/charts/master/logo/ibm-mq-icon.svg?sanitize=true" width="100" alt="IBM MQ logo" />
## Overview
Run [IBM® MQ](http://www-03.ibm.com/software/products/en/ibm-mq) in a container.
@@ -46,12 +44,12 @@ For issues relating specifically to the container image or Helm chart, please us
The Dockerfiles and associated code and scripts are licensed under the [Apache License 2.0](http://www.apache.org/licenses/LICENSE-2.0.html).
Licenses for the products installed within the images are as follows:
- [IBM MQ Advanced for Developers](http://www14.software.ibm.com/cgi-bin/weblap/lap.pl?la_formnum=Z125-3301-14&li_formnum=L-APIG-AVCJ4S) (International License Agreement for Non-Warranted Programs). This license may be viewed from an image using the `LICENSE=view` environment variable as described above or by following the link above.
- [IBM MQ Advanced](http://www14.software.ibm.com/cgi-bin/weblap/lap.pl?la_formnum=Z125-3301-14&li_formnum=L-APIG-AZYF4X) (International Program License Agreement). This license may be viewed from an image using the `LICENSE=view` environment variable as described above or by following the link above.
- [IBM MQ Advanced for Developers](http://www14.software.ibm.com/cgi-bin/weblap/lap.pl?la_formnum=Z125-3301-14&li_formnum=L-APIG-BBZHCQ) (International License Agreement for Non-Warranted Programs). This license may be viewed from an image using the `LICENSE=view` environment variable as described above or by following the link above.
- [IBM MQ Advanced](http://www14.software.ibm.com/cgi-bin/weblap/lap.pl?la_formnum=Z125-3301-14&li_formnum=L-APIG-BBSHJL) (International Program License Agreement). This license may be viewed from an image using the `LICENSE=view` environment variable as described above or by following the link above.
- License information for Ubuntu packages may be found in `/usr/share/doc/${package}/copyright`
Note: The IBM MQ Advanced for Developers license does not permit further distribution and the terms restrict usage to a developer machine.
## Copyright
© Copyright IBM Corporation 2015, 2018
© Copyright IBM Corporation 2015, 2019

23
buildah/Dockerfile Normal file
View File

@@ -0,0 +1,23 @@
# © Copyright IBM Corporation 2019
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Fedora includes more recent versions of buildah (need buildah V1.7 to get
# multi-stage builds to work properly)
FROM docker.io/fedora:29
RUN yum install -y buildah
COPY build.sh /usr/local/bin/build
RUN chmod +x /usr/local/bin/build
ENV STORAGE_DRIVER=vfs
ENV BUILDAH_ISOLATION=chroot
ENTRYPOINT ["build"]

41
buildah/build.sh Normal file
View File

@@ -0,0 +1,41 @@
#!/bin/bash
# -*- mode: sh -*-
# © Copyright IBM Corporation 2019
#
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Build a container image from a Dockerfile using Buildah
# If the Docker socket is available, the image will be pushed to Docker once built.
readonly IMAGE=$1:$2
shift
shift
readonly SRC="/src"
readonly OCI_DIR="/var/oci"
echo "****************************************"
echo " Inside the Buildah container"
echo "****************************************"
set -ex
# Build using the supplied options. Always pass the source directory in, and
# use it as the build context
buildah build-using-dockerfile --tag ${IMAGE} --volume /src:/src "$@" /src
if [ -e ${OCI_DIR} ]; then
buildah push ${IMAGE} oci-archive:${OCI_DIR}/${IMAGE}
fi
if [ -e /var/run/docker.sock ]; then
buildah push ${IMAGE} docker-daemon:${IMAGE}
fi

51
buildah/mq-buildah Executable file
View File

@@ -0,0 +1,51 @@
#!/bin/bash
# -*- mode: sh -*-
# © Copyright IBM Corporation 2019
#
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Build and run a container image with Buildah installed
if [ $# -eq 0 ]; then
echo "Usage: $0 imageName imageTag buildah-options"
exit 1
fi
readonly SCRIPT_DIR="$( cd "$(dirname "$0")" ; pwd -P )"
readonly DIR=${SCRIPT_DIR}/..
readonly BUILDAH_IMAGE=mq-buildah
RUN_OPTS="--volume ${DIR}:/src --cap-add SYS_ADMIN --rm --interactive --tty"
CMD="docker"
# If Docker is installed, then map the Docker socket into the container, to
# allow buildah to push the resulting image into Docker's image store
# WARNING: This means that the buildah container can do anything the "docker"
# can do.
command -v docker
if [[ $? == 0 && -e /var/run/docker.sock ]]; then
RUN_OPTS="${RUN_OPTS} --volume /var/run/docker.sock:/var/run/docker.sock"
fi
command -v podman
if [[ $? -eq 0 ]]; then
CMD="podman"
OCI_DIR="/tmp/mq-buildah"
mkdir -p ${OCI_DIR}
RUN_OPTS="${RUN_OPTS} --volume ${OCI_DIR}:/var/oci"
echo "Image archives will be written to ${OCI_DIR}"
fi
${CMD} build --tag ${BUILDAH_IMAGE} --file ${SCRIPT_DIR}/Dockerfile ${SCRIPT_DIR}
set -x
${CMD} run ${RUN_OPTS} ${BUILDAH_IMAGE} "$@"

View File

@@ -1,5 +1,5 @@
/*
© Copyright IBM Corporation 2017
© Copyright IBM Corporation 2017, 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -41,7 +41,7 @@ func queueManagerHealthy() (bool, error) {
return false, err
}
fmt.Printf("%s", out)
if !strings.Contains(string(out), "(RUNNING)") {
if !strings.Contains(string(out), "(RUNNING)") && !strings.Contains(string(out), "(RUNNING AS STANDBY)") && !strings.Contains(string(out), "(STARTING)") {
return false, nil
}
return true, nil

View File

@@ -1,5 +1,5 @@
/*
© Copyright IBM Corporation 2017, 2018
© Copyright IBM Corporation 2017, 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -22,6 +22,7 @@ import (
"net"
"os"
"github.com/ibm-messaging/mq-container/internal/name"
"github.com/ibm-messaging/mq-container/internal/ready"
)
@@ -31,14 +32,25 @@ func main() {
if !r || err != nil {
os.Exit(1)
}
// Check if the queue manager has a running listener
conn, err := net.Dial("tcp", "127.0.0.1:1414")
name, err := name.GetQueueManagerName()
if err != nil {
fmt.Println(err)
os.Exit(1)
}
err = conn.Close()
if err != nil {
fmt.Println(err)
// Check if the queue manager has a running listener
if standby, _ := ready.IsRunningAsStandbyQM(name); !standby {
conn, err := net.Dial("tcp", "127.0.0.1:1414")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
err = conn.Close()
if err != nil {
fmt.Println(err)
}
} else {
fmt.Printf("Detected queue manager running in standby mode")
os.Exit(10)
}
}

View File

@@ -1,68 +0,0 @@
/*
© Copyright IBM Corporation 2017, 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"runtime"
"strings"
containerruntime "github.com/ibm-messaging/mq-container/internal/containerruntime"
"github.com/ibm-messaging/mq-container/internal/user"
)
func logContainerDetails() {
log.Printf("CPU architecture: %v", runtime.GOARCH)
kv, err := containerruntime.GetKernelVersion()
if err == nil {
log.Printf("Linux kernel version: %v", kv)
}
cr, err := containerruntime.GetContainerRuntime()
if err == nil {
log.Printf("Container runtime: %v", cr)
}
bi, err := containerruntime.GetBaseImage()
if err == nil {
log.Printf("Base image: %v", bi)
}
u, err := user.GetUser()
if err == nil {
if len(u.SupplementalGID) == 0 {
log.Printf("Running as user ID %v (%v) with primary group %v", u.UID, u.Name, u.PrimaryGID)
} else {
log.Printf("Running as user ID %v (%v) with primary group %v, and supplementary groups %v", u.UID, u.Name, u.PrimaryGID, strings.Join(u.SupplementalGID, ","))
}
}
caps, err := containerruntime.GetCapabilities()
capLogged := false
if err == nil {
for k, v := range caps {
if len(v) > 0 {
log.Printf("Capabilities (%s set): %v", strings.ToLower(k), strings.Join(v, ","))
capLogged = true
}
}
if !capLogged {
log.Print("Capabilities: none")
}
} else {
log.Errorf("Error getting capabilities: %v", err)
}
sc, err := containerruntime.GetSeccomp()
if err == nil {
log.Printf("seccomp enforcing mode: %v", sc)
}
log.Printf("Process security attributes: %v", containerruntime.GetSecurityAttributes())
}

View File

@@ -23,7 +23,9 @@ import (
"syscall"
"github.com/ibm-messaging/mq-container/internal/command"
"github.com/ibm-messaging/mq-container/internal/containerruntimelogger"
"github.com/ibm-messaging/mq-container/internal/logger"
"github.com/ibm-messaging/mq-container/internal/mqtemplate"
"github.com/ibm-messaging/mq-container/internal/name"
)
@@ -90,7 +92,7 @@ func configureLogger() error {
func configureWeb(qmName string) error {
out := "/etc/mqm/web/installations/Installation1/angular.persistence/admin.json"
return processTemplateFile("/etc/mqm/admin.json.tpl", out, map[string]string{"QueueManagerName": qmName})
return mqtemplate.ProcessTemplateFile("/etc/mqm/admin.json.tpl", out, map[string]string{"QueueManagerName": qmName}, log)
}
func logTerminationf(format string, args ...interface{}) {
@@ -117,7 +119,11 @@ func doMain() error {
return err
}
logContainerDetails()
err = containerruntimelogger.LogContainerDetails(log)
if err != nil {
logTermination(err)
return err
}
adminPassword, set := os.LookupEnv("MQ_ADMIN_PASSWORD")
if set {
@@ -147,14 +153,6 @@ func doMain() error {
logTerminationf("Error getting queue manager name: %v", err)
return err
}
ks, set := os.LookupEnv("MQ_TLS_KEYSTORE")
if set {
err = configureTLS(name, ks, os.Getenv("MQ_TLS_PASSPHRASE"))
if err != nil {
logTerminationf("Error configuring TLS: %v", err)
return err
}
}
err = configureWeb(name)
if err != nil {

View File

@@ -1,5 +1,5 @@
/*
© Copyright IBM Corporation 2018
© Copyright IBM Corporation 2018, 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -17,6 +17,8 @@ package main
import (
"os"
"github.com/ibm-messaging/mq-container/internal/mqtemplate"
)
func updateMQSC(appPasswordRequired bool) error {
@@ -30,7 +32,7 @@ func updateMQSC(appPasswordRequired bool) error {
if os.Getenv("MQ_DEV") == "true" {
const mqscTemplate string = mqsc + ".tpl"
// Re-configure channel if app password not set
err := processTemplateFile(mqsc+".tpl", mqsc, map[string]string{"ChckClnt": checkClient})
err := mqtemplate.ProcessTemplateFile(mqsc+".tpl", mqsc, map[string]string{"ChckClnt": checkClient}, log)
if err != nil {
return err
}

View File

@@ -1,164 +0,0 @@
/*
© Copyright IBM Corporation 2018
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"fmt"
"os"
"path/filepath"
"github.com/ibm-messaging/mq-container/internal/command"
)
func configureWebTLS(cms *KeyStore) error {
dir := "/run/runmqdevserver/tls"
ks := NewJKSKeyStore(filepath.Join(dir, "key.jks"), cms.Password)
ts := NewJKSKeyStore(filepath.Join(dir, "trust.jks"), cms.Password)
log.Debug("Creating key store")
err := ks.Create()
if err != nil {
return err
}
log.Debug("Creating trust store")
err = ts.Create()
if err != nil {
return err
}
log.Debug("Importing keys")
err = ks.Import(cms.Filename, cms.Password)
if err != nil {
return err
}
webConfigDir := "/etc/mqm/web/installations/Installation1/servers/mqweb"
tlsConfig := filepath.Join(webConfigDir, "tls.xml")
newTLSConfig := filepath.Join(webConfigDir, "tls-dev.xml")
err = os.Remove(tlsConfig)
if err != nil {
return err
}
// we symlink here to prevent issues on restart
err = os.Symlink(newTLSConfig, tlsConfig)
if err != nil {
return err
}
mqmUID, mqmGID, err := command.LookupMQM()
if err != nil {
log.Error(err)
return err
}
err = os.Chown(tlsConfig, mqmUID, mqmGID)
if err != nil {
log.Error(err)
return err
}
return nil
}
func configureTLS(qmName string, inputFile string, passPhrase string) error {
log.Debug("Configuring TLS")
_, err := os.Stat(inputFile)
if err != nil {
return err
}
// TODO: Use a persisted file (on the volume) instead?
dir := "/run/runmqdevserver/tls"
keyFile := filepath.Join(dir, "key.kdb")
_, err = os.Stat(dir)
if err != nil {
if os.IsNotExist(err) {
// #nosec G301
err = os.MkdirAll(dir, 0770)
if err != nil {
return err
}
mqmUID, mqmGID, err := command.LookupMQM()
if err != nil {
log.Error(err)
return err
}
err = os.Chown(dir, mqmUID, mqmGID)
if err != nil {
log.Error(err)
return err
}
} else {
return err
}
}
cms := NewCMSKeyStore(keyFile, passPhrase)
err = cms.Create()
if err != nil {
return err
}
err = cms.CreateStash()
if err != nil {
return err
}
err = cms.Import(inputFile, passPhrase)
if err != nil {
return err
}
labels, err := cms.GetCertificateLabels()
if err != nil {
return err
}
if len(labels) == 0 {
return fmt.Errorf("unable to find certificate label")
}
log.Debugf("Renaming certificate from %v", labels[0])
const newLabel string = "devcert"
err = cms.RenameCertificate(labels[0], newLabel)
if err != nil {
return err
}
var sslCipherSpec string
if os.Getenv("MQ_DEV") == "true" {
sslCipherSpec = "TLS_RSA_WITH_AES_128_CBC_SHA256"
} else {
sslCipherSpec = "' '"
}
const mqsc string = "/etc/mqm/20-dev-tls.mqsc"
const mqscTemplate string = mqsc + ".tpl"
err = processTemplateFile(mqscTemplate, mqsc, map[string]string{
"SSLKeyR": filepath.Join(dir, "key"),
"CertificateLabel": newLabel,
"SSLCipherSpec": sslCipherSpec,
})
if err != nil {
return err
}
err = configureWebTLS(cms)
if err != nil {
return err
}
return nil
}

View File

@@ -1,5 +1,5 @@
/*
© Copyright IBM Corporation 2017, 2018
© Copyright IBM Corporation 2017, 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -17,15 +17,13 @@ package main
import (
"os"
"path/filepath"
"runtime"
"syscall"
"github.com/ibm-messaging/mq-container/internal/command"
)
func createVolume(path string) error {
dataPath := filepath.Join(path, "data")
func createVolume(dataPath string) error {
fi, err := os.Stat(dataPath)
if err != nil {
if os.IsNotExist(err) {

View File

@@ -23,6 +23,7 @@ import (
"os"
"os/exec"
"path/filepath"
"strings"
"sync"
"github.com/ibm-messaging/mq-container/internal/command"
@@ -66,7 +67,7 @@ func formatSimple(datetime string, message string) string {
// mirrorSystemErrorLogs starts a goroutine to mirror the contents of the MQ system error logs
func mirrorSystemErrorLogs(ctx context.Context, wg *sync.WaitGroup, mf mirrorFunc) (chan error, error) {
// Always use the JSON log as the source
return mirrorLog(ctx, wg, "/var/mqm/errors/AMQERR01.json", false, mf)
return mirrorLog(ctx, wg, "/var/mqm/errors/AMQERR01.json", false, mf, false)
}
// mirrorQueueManagerErrorLogs starts a goroutine to mirror the contents of the MQ queue manager error logs
@@ -78,7 +79,7 @@ func mirrorQueueManagerErrorLogs(ctx context.Context, wg *sync.WaitGroup, name s
return nil, err
}
f := filepath.Join(mqini.GetErrorLogDirectory(qm), "AMQERR01.json")
return mirrorLog(ctx, wg, f, fromStart, mf)
return mirrorLog(ctx, wg, f, fromStart, mf, true)
}
func getDebug() bool {
@@ -99,21 +100,35 @@ func configureLogger(name string) (mirrorFunc, error) {
if err != nil {
return nil, err
}
return log.LogDirect, nil
return func(msg string, isQMLog bool) bool {
obj, err := processLogMessage(msg)
if err == nil && isQMLog && filterQMLogMessage(obj) {
return false
}
if err != nil {
log.Printf("Failed to unmarshall JSON - %v", msg)
} else {
fmt.Println(msg)
}
return true
}, nil
case "basic":
log, err = logger.NewLogger(os.Stderr, d, false, name)
if err != nil {
return nil, err
}
return func(msg string) {
return func(msg string, isQMLog bool) bool {
// Parse the JSON message, and print a simplified version
var obj map[string]interface{}
err := json.Unmarshal([]byte(msg), &obj)
obj, err := processLogMessage(msg)
if err == nil && isQMLog && filterQMLogMessage(obj) {
return false
}
if err != nil {
fmt.Printf("Failed to Unmarshall JSON - %v", err)
log.Printf("Failed to unmarshall JSON - %v", err)
} else {
fmt.Printf(formatSimple(obj["ibm_datetime"].(string), obj["message"].(string)))
}
return true
}, nil
default:
log, err = logger.NewLogger(os.Stdout, d, false, name)
@@ -124,6 +139,20 @@ func configureLogger(name string) (mirrorFunc, error) {
}
}
func processLogMessage(msg string) (map[string]interface{}, error) {
var obj map[string]interface{}
err := json.Unmarshal([]byte(msg), &obj)
return obj, err
}
func filterQMLogMessage(obj map[string]interface{}) bool {
hostname, err := os.Hostname()
if os.Getenv("MQ_MULTI_INSTANCE") == "true" && err == nil && !strings.Contains(obj["host"].(string), hostname) {
return true
}
return false
}
func logDiagnostics() {
log.Debug("--- Start Diagnostics ---")
@@ -138,11 +167,20 @@ func logDiagnostics() {
out, _, _ = command.Run("ls", "-l", "/mnt/mqm/data")
log.Debugf("/mnt/mqm/data:\n%s", out)
// #nosec G104
out, _, _ = command.Run("ls", "-l", "/mnt/mqm-log/log")
log.Debugf("/mnt/mqm-log/log:\n%s", out)
// #nosec G104
out, _, _ = command.Run("ls", "-l", "/mnt/mqm-data/qmgrs")
log.Debugf("/mnt/mqm-data/qmgrs:\n%s", out)
// #nosec G104
out, _, _ = command.Run("ls", "-l", "/var/mqm")
log.Debugf("/var/mqm:\n%s", out)
// #nosec G104
out, _, _ = command.Run("ls", "-l", "/var/mqm/errors")
log.Debugf("/var/mqm/errors:\n%s", out)
// #nosec G104
out, _, _ = command.Run("ls", "-l", "/etc/mqm")
log.Debugf("/etc/mqm:\n%s", out)
// Print out summary of any FDCs
// #nosec G204

View File

@@ -24,9 +24,11 @@ import (
"os"
"sync"
"github.com/ibm-messaging/mq-container/internal/containerruntimelogger"
"github.com/ibm-messaging/mq-container/internal/metrics"
"github.com/ibm-messaging/mq-container/internal/name"
"github.com/ibm-messaging/mq-container/internal/ready"
"github.com/ibm-messaging/mq-container/internal/tls"
)
func doMain() error {
@@ -45,7 +47,7 @@ func doMain() error {
// Check whether they only want debug info
if *infoFlag {
logVersionInfo()
err = logContainerDetails()
err = containerruntimelogger.LogContainerDetails(log)
if err != nil {
log.Printf("Error displaying container details: %v", err)
}
@@ -86,18 +88,29 @@ func doMain() error {
collectDiagOnFail = true
if *devFlag == false {
err = logContainerDetails()
err = containerruntimelogger.LogContainerDetails(log)
if err != nil {
logTermination(err)
return err
}
}
err = createVolume("/mnt/mqm")
err = createVolume("/mnt/mqm/data")
if err != nil {
logTermination(err)
return err
}
err = createVolume("/mnt/mqm-log/log")
if err != nil {
logTermination(err)
return err
}
err = createVolume("/mnt/mqm-data/qmgrs")
if err != nil {
logTermination(err)
return err
}
err = createDirStructure()
if err != nil {
logTermination(err)
@@ -112,7 +125,19 @@ func doMain() error {
// Print out versioning information
logVersionInfo()
err = postInit(name)
keylabel, cmsDB, p12Trust, _, err := tls.ConfigureTLSKeystores(keyDir, trustDir, keyStoreDir)
if err != nil {
logTermination(err)
return err
}
err = configureTLS(keylabel, cmsDB, *devFlag)
if err != nil {
logTermination(err)
return err
}
err = postInit(name, keylabel, p12Trust)
if err != nil {
logTermination(err)
return err
@@ -149,15 +174,17 @@ func doMain() error {
logTermination(err)
return err
}
err = startQueueManager()
err = startQueueManager(name)
if err != nil {
logTermination(err)
return err
}
err = configureQueueManager()
if err != nil {
logTermination(err)
return err
if standby, _ := ready.IsRunningAsStandbyQM(name); !standby {
err = configureQueueManager()
if err != nil {
logTermination(err)
return err
}
}
enableMetrics := os.Getenv("MQ_ENABLE_METRICS")

View File

@@ -1,5 +1,5 @@
/*
© Copyright IBM Corporation 2018
© Copyright IBM Corporation 2018, 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -49,16 +49,17 @@ func waitForFile(ctx context.Context, path string) (os.FileInfo, error) {
}
}
type mirrorFunc func(msg string)
type mirrorFunc func(msg string, isQMLog bool) bool
// mirrorAvailableMessages prints lines from the file, until no more are available
func mirrorAvailableMessages(f *os.File, mf mirrorFunc) {
func mirrorAvailableMessages(f *os.File, mf mirrorFunc, isQMLog bool) {
scanner := bufio.NewScanner(f)
count := 0
for scanner.Scan() {
t := scanner.Text()
mf(t)
count++
if mf(t, isQMLog) {
count++
}
}
if count > 0 {
log.Debugf("Mirrored %v log entries from %v", count, f.Name())
@@ -73,7 +74,7 @@ func mirrorAvailableMessages(f *os.File, mf mirrorFunc) {
// mirrorLog tails the specified file, and logs each line to stdout.
// This is useful for usability, as the container console log can show
// messages from the MQ error logs.
func mirrorLog(ctx context.Context, wg *sync.WaitGroup, path string, fromStart bool, mf mirrorFunc) (chan error, error) {
func mirrorLog(ctx context.Context, wg *sync.WaitGroup, path string, fromStart bool, mf mirrorFunc, isQMLog bool) (chan error, error) {
errorChannel := make(chan error, 1)
var offset int64 = -1
var f *os.File
@@ -147,7 +148,7 @@ func mirrorLog(ctx context.Context, wg *sync.WaitGroup, path string, fromStart b
closing := false
for {
// If there's already data there, mirror it now.
mirrorAvailableMessages(f, mf)
mirrorAvailableMessages(f, mf, isQMLog)
// Wait for the new log file (after rotation)
newFI, err := waitForFile(ctx, path)
if err != nil {
@@ -161,7 +162,7 @@ func mirrorLog(ctx context.Context, wg *sync.WaitGroup, path string, fromStart b
// log rotation happens before we can open the new file, then we
// could skip all those messages. This could happen with a very small
// MQ error log size.
mirrorAvailableMessages(f, mf)
mirrorAvailableMessages(f, mf, isQMLog)
err = f.Close()
if err != nil {
log.Errorf("Unable to close mirror file handle: %v", err)
@@ -176,7 +177,7 @@ func mirrorLog(ctx context.Context, wg *sync.WaitGroup, path string, fromStart b
}
fi = newFI
// Don't seek this time, because we know it's a new file
mirrorAvailableMessages(f, mf)
mirrorAvailableMessages(f, mf, isQMLog)
}
select {
case <-ctx.Done():

View File

@@ -1,5 +1,5 @@
/*
© Copyright IBM Corporation 2018
© Copyright IBM Corporation 2018, 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -41,9 +41,10 @@ func TestMirrorLogWithoutRotation(t *testing.T) {
count := 0
ctx, cancel := context.WithCancel(context.Background())
var wg sync.WaitGroup
_, err = mirrorLog(ctx, &wg, tmp.Name(), true, func(msg string) {
_, err = mirrorLog(ctx, &wg, tmp.Name(), true, func(msg string, isQMLog bool) bool {
count++
})
return true
}, false)
if err != nil {
t.Fatal(err)
}
@@ -82,9 +83,10 @@ func TestMirrorLogWithRotation(t *testing.T) {
count := 0
ctx, cancel := context.WithCancel(context.Background())
var wg sync.WaitGroup
_, err = mirrorLog(ctx, &wg, tmp.Name(), true, func(msg string) {
_, err = mirrorLog(ctx, &wg, tmp.Name(), true, func(msg string, isQMLog bool) bool {
count++
})
return true
}, false)
if err != nil {
t.Fatal(err)
}
@@ -135,9 +137,10 @@ func testMirrorLogExistingFile(t *testing.T, newQM bool) int {
count := 0
ctx, cancel := context.WithCancel(context.Background())
var wg sync.WaitGroup
_, err = mirrorLog(ctx, &wg, tmp.Name(), newQM, func(msg string) {
_, err = mirrorLog(ctx, &wg, tmp.Name(), newQM, func(msg string, isQMLog bool) bool {
count++
})
return true
}, false)
if err != nil {
t.Fatal(err)
}
@@ -179,8 +182,9 @@ func TestMirrorLogCancelWhileWaiting(t *testing.T) {
cancel()
wg.Wait()
}()
_, err := mirrorLog(ctx, &wg, "fake.log", true, func(msg string) {
})
_, err := mirrorLog(ctx, &wg, "fake.log", true, func(msg string, isQMLog bool) bool {
return true
}, false)
if err != nil {
t.Error(err)
}

View File

@@ -1,7 +1,5 @@
// +build mqdev
/*
© Copyright IBM Corporation 2018
© Copyright IBM Corporation 2018, 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -19,23 +17,26 @@ package main
import (
"os"
"github.com/ibm-messaging/mq-container/internal/tls"
)
// postInit is run after /var/mqm is set up
// This version of postInit is only included as part of the MQ Advanced for Developers build
func postInit(name string) error {
disable := os.Getenv("MQ_DISABLE_WEB_CONSOLE")
if disable != "true" && disable != "1" {
// Configure the web server (if installed)
err := configureWebServer()
func postInit(name, keylabel string, p12Trust tls.KeyStoreData) error {
enableWebServer := os.Getenv("MQ_ENABLE_EMBEDDED_WEB_SERVER")
if enableWebServer == "true" || enableWebServer == "1" {
// Configure the web server (if enabled)
keystore, err := configureWebServer(keylabel, p12Trust)
if err != nil {
return err
}
// Start the web server, in the background (if installed)
// WARNING: No error handling or health checking available for the web server,
// which is why it's limited to use with MQ Advanced for Developers only
// WARNING: No error handling or health checking available for the web server
go func() {
startWebServer()
err = startWebServer(keystore, p12Trust.Password)
if err != nil {
log.Printf("Error starting web server: %v", err)
}
}()
}
return nil

View File

@@ -16,7 +16,8 @@ limitations under the License.
package main
import (
"io"
"bytes"
"fmt"
"io/ioutil"
"os"
"os/exec"
@@ -24,11 +25,14 @@ import (
"strings"
"github.com/ibm-messaging/mq-container/internal/command"
containerruntime "github.com/ibm-messaging/mq-container/internal/containerruntime"
"github.com/ibm-messaging/mq-container/internal/mqscredact"
"github.com/ibm-messaging/mq-container/internal/ready"
)
// createDirStructure creates the default MQ directory structure under /var/mqm
func createDirStructure() error {
out, _, err := command.Run("/opt/mqm/bin/crtmqdir", "-f", "-s")
out, _, err := command.Run("/opt/mqm/bin/crtmqdir", "-f", "-a")
if err != nil {
log.Printf("Error creating directory structure: %v\n", string(out))
return err
@@ -38,20 +42,48 @@ func createDirStructure() error {
}
// createQueueManager creates a queue manager, if it doesn't already exist.
// It returns true if one was created, or false if one already existed
// It returns true if one was created (or a standby was created), or false if one already existed
func createQueueManager(name string) (bool, error) {
log.Printf("Creating queue manager %v", name)
out, rc, err := command.Run("crtmqm", "-q", "-p", "1414", name)
// Run 'dspmqinf' to check if 'mqs.ini' configuration file exists
// If command succeeds, the queue manager (or standby queue manager) has already been created
_, _, err := command.Run("dspmqinf", name)
if err == nil {
log.Printf("Detected existing queue manager %v", name)
return false, nil
}
mounts, err := containerruntime.GetMounts()
if err != nil {
// 8=Queue manager exists, which is fine
if rc == 8 {
log.Printf("Detected existing queue manager %v", name)
return false, nil
}
log.Printf("crtmqm returned %v", rc)
log.Println(string(out))
log.Printf("Error getting mounts for queue manager")
return false, err
}
// Check if 'qm.ini' configuration file exists for the queue manager
// TODO : handle possible race condition - use a file lock?
dataDir := getQueueManagerDataDir(mounts, name)
_, err = os.Stat(filepath.Join(dataDir, "qm.ini"))
if err != nil {
// If 'qm.ini' is not found - run 'crtmqm' to create a new queue manager
args := getCreateQueueManagerArgs(mounts, name)
out, rc, err := command.Run("crtmqm", args...)
if err != nil {
log.Printf("Error %v creating queue manager: %v", rc, string(out))
return false, err
}
} else {
// If 'qm.ini' is found - run 'addmqinf' to create a standby queue manager with existing configuration
args := getCreateStandbyQueueManagerArgs(name)
out, rc, err := command.Run("addmqinf", args...)
if err != nil {
log.Printf("Error %v creating standby queue manager: %v", rc, string(out))
return false, err
}
log.Println("Created standby queue manager")
return true, nil
}
log.Println("Created queue manager")
return true, nil
}
@@ -68,10 +100,15 @@ func updateCommandLevel() error {
return nil
}
func startQueueManager() error {
func startQueueManager(name string) error {
log.Println("Starting queue manager")
out, rc, err := command.Run("strmqm")
out, rc, err := command.Run("strmqm", "-x", name)
if err != nil {
// 30=standby queue manager started, which is fine
if rc == 30 {
log.Printf("Started standby queue manager")
return nil
}
log.Printf("Error %v starting queue manager: %v", rc, string(out))
return err
}
@@ -86,43 +123,49 @@ func configureQueueManager() error {
log.Println(err)
return err
}
for _, file := range files {
if strings.HasSuffix(file.Name(), ".mqsc") {
abs := filepath.Join(configDir, file.Name())
// #nosec G204
verify := exec.Command("runmqsc", "-v", "-e")
// #nosec G204 - command is fixed, no injection vector
cmd := exec.Command("runmqsc")
stdin, err := cmd.StdinPipe()
// Read mqsc file into variable
// #nosec G304 - filename variable is derived from contents of 'configDir' which is a defined constant
mqsc, err := ioutil.ReadFile(abs)
if err != nil {
log.Println(err)
return err
log.Printf("Error reading file %v: %v", abs, err)
continue
}
// Open the MQSC file for reading
// #nosec G304
f, err := os.Open(abs)
// Write mqsc to buffer
var buffer bytes.Buffer
_, err = buffer.Write(mqsc)
if err != nil {
log.Printf("Error opening %v: %v", abs, err)
log.Printf("Error writing MQSC file %v to buffer: %v", abs, err)
continue
}
// Copy the contents to stdin of the runmqsc process
_, err = io.Copy(stdin, f)
verifyBuffer := buffer
// Buffer mqsc to stdin of runmqsc
cmd.Stdin = &buffer
verify.Stdin = &verifyBuffer
// Verify the MQSC commands
out, err := verify.CombinedOutput()
if err != nil {
log.Errorf("Error reading %v: %v", abs, err)
log.Errorf("Error verifying MQSC file %v (%v):\n\t%v", file.Name(), err, formatMQSCOutput(string(out)))
return fmt.Errorf("Error verifying MQSC file %v (%v):\n\t%v", file.Name(), err, formatMQSCOutput(string(out)))
}
err = f.Close()
// Run runmqsc command
out, err = cmd.CombinedOutput()
if err != nil {
log.Errorf("Failed to close MQSC file handle: %v", err)
log.Errorf("Error running MQSC file %v (%v):\n\t%v", file.Name(), err, formatMQSCOutput(string(out)))
continue
} else {
// Print the runmqsc output, adding tab characters to make it more readable as part of the log
log.Printf("Output for \"runmqsc\" with %v:\n\t%v", abs, formatMQSCOutput(string(out)))
}
err = stdin.Close()
if err != nil {
log.Errorf("Failed to close MQSC stdin: %v", err)
}
// Run the command and wait for completion
out, err := cmd.CombinedOutput()
if err != nil {
log.Errorf("Error running MQSC file %v (%v):\n\t%v", file.Name(), err, strings.Replace(string(out), "\n", "\n\t", -1))
}
// Print the runmqsc output, adding tab characters to make it more readable as part of the log
log.Printf("Output for \"runmqsc\" with %v:\n\t%v", abs, strings.Replace(string(out), "\n", "\n\t", -1))
}
}
return nil
@@ -130,11 +173,77 @@ func configureQueueManager() error {
func stopQueueManager(name string) error {
log.Println("Stopping queue manager")
out, _, err := command.Run("endmqm", "-w", "-r", name)
isStandby, err := ready.IsRunningAsStandbyQM(name)
if err != nil {
log.Printf("Error stopping queue manager: %v", string(out))
log.Printf("Error getting status for queue manager %v: ", name, err.Error())
return err
}
log.Println("Stopped queue manager")
args := []string{"-w", "-r", name}
if os.Getenv("MQ_MULTI_INSTANCE") == "true" {
if isStandby {
args = []string{"-x", name}
} else {
args = []string{"-s", "-w", "-r", name}
}
}
out, rc, err := command.Run("endmqm", args...)
if err != nil {
log.Printf("Error %v stopping queue manager: %v", rc, string(out))
return err
}
if isStandby {
log.Printf("Stopped standby queue manager")
} else {
log.Println("Stopped queue manager")
}
return nil
}
func formatMQSCOutput(out string) string {
// redact sensitive information
out, _ = mqscredact.Redact(out)
// add tab characters to make it more readable as part of the log
return strings.Replace(string(out), "\n", "\n\t", -1)
}
func isStandbyQueueManager(name string) (bool, error) {
out, rc, err := command.Run("dspmq", "-n", "-m", name)
if err != nil {
log.Printf("Error %v getting status for queue manager %v: %v", rc, name, string(out))
return false, err
}
if strings.Contains(string(out), "(RUNNING AS STANDBY)") {
return true, nil
}
return false, nil
}
func getQueueManagerDataDir(mounts map[string]string, name string) string {
dataDir := filepath.Join("/var/mqm/qmgrs", name)
if _, ok := mounts["/mnt/mqm-data"]; ok {
dataDir = filepath.Join("/mnt/mqm-data/qmgrs", name)
}
return dataDir
}
func getCreateQueueManagerArgs(mounts map[string]string, name string) []string {
args := []string{"-q", "-p", "1414"}
if _, ok := mounts["/mnt/mqm-log"]; ok {
args = append(args, "-ld", "/mnt/mqm-log/log")
}
if _, ok := mounts["/mnt/mqm-data"]; ok {
args = append(args, "-md", "/mnt/mqm-data/qmgrs")
}
args = append(args, name)
return args
}
func getCreateStandbyQueueManagerArgs(name string) []string {
args := []string{"-s", "QueueManager"}
args = append(args, "-v", fmt.Sprintf("Name=%v", name))
args = append(args, "-v", fmt.Sprintf("Directory=%v", name))
args = append(args, "-v", "Prefix=/var/mqm")
args = append(args, "-v", fmt.Sprintf("DataPath=/mnt/mqm-data/qmgrs/%v", name))
return args
}

166
cmd/runmqserver/tls.go Normal file
View File

@@ -0,0 +1,166 @@
/*
© Copyright IBM Corporation 2018, 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"fmt"
"os"
"path/filepath"
"strings"
"github.com/ibm-messaging/mq-container/internal/command"
"github.com/ibm-messaging/mq-container/internal/keystore"
"github.com/ibm-messaging/mq-container/internal/mqtemplate"
"github.com/ibm-messaging/mq-container/internal/tls"
)
// Location to store the keystores
const keyStoreDir = "/run/runmqserver/tls/"
// KeyDir is the location of the certificate keys to import
const keyDir = "/etc/mqm/pki/keys"
// TrustDir is the location of the Certifates to add
const trustDir = "/etc/mqm/pki/trust"
// configureWebTLS configures TLS for Web Console
func configureWebTLS(label string) error {
// Return immediately if we have no certificate to use as identity
if label == "" && os.Getenv("MQ_GENERATE_CERTIFICATE_HOSTNAME") == "" {
return nil
}
webConfigDir := "/etc/mqm/web/installations/Installation1/servers/mqweb"
tls := "tls.xml"
tlsConfig := filepath.Join(webConfigDir, tls)
newTLSConfig := filepath.Join(webConfigDir, tls+".tpl")
err := os.Remove(tlsConfig)
if err != nil {
return fmt.Errorf("Could not delete file %s: %v", tlsConfig, err)
}
// we symlink here to prevent issues on restart
err = os.Symlink(newTLSConfig, tlsConfig)
if err != nil {
return fmt.Errorf("Could not create symlink %s->%s: %v", newTLSConfig, tlsConfig, err)
}
mqmUID, mqmGID, err := command.LookupMQM()
if err != nil {
return fmt.Errorf("Could not find mqm user or group: %v", err)
}
err = os.Chown(tlsConfig, mqmUID, mqmGID)
if err != nil {
return fmt.Errorf("Could change ownership of %s to mqm: %v", tlsConfig, err)
}
return nil
}
// configureTLSDev configures TLS for developer defaults
func configureTLSDev() error {
const mqsc string = "/etc/mqm/20-dev-tls.mqsc"
const mqscTemplate string = mqsc + ".tpl"
const sslCipherSpec string = "TLS_RSA_WITH_AES_128_CBC_SHA256"
if os.Getenv("MQ_DEV") == "true" {
err := mqtemplate.ProcessTemplateFile(mqscTemplate, mqsc, map[string]string{
"SSLCipherSpec": sslCipherSpec,
}, log)
if err != nil {
return err
}
} else {
_, err := os.Stat(mqsc)
if !os.IsNotExist(err) {
err = os.Remove(mqsc)
if err != nil {
log.Errorf("Error removing file %s: %v", mqsc, err)
return err
}
}
}
return nil
}
// configureTLS configures TLS for queue manager
func configureTLS(certLabel string, cmsKeystore tls.KeyStoreData, devmode bool) error {
log.Debug("Configuring TLS")
const mqsc string = "/etc/mqm/15-tls.mqsc"
const mqscTemplate string = mqsc + ".tpl"
err := mqtemplate.ProcessTemplateFile(mqscTemplate, mqsc, map[string]string{
"SSLKeyR": strings.TrimSuffix(cmsKeystore.Keystore.Filename, ".kdb"),
"CertificateLabel": certLabel,
}, log)
if err != nil {
return err
}
if devmode && certLabel != "" {
err = configureTLSDev()
if err != nil {
return err
}
}
return nil
}
// configureSSOTLS configures MQ Console TLS for Single Sign-On
func configureSSOTLS(p12TrustStore tls.KeyStoreData) (string, error) {
// TODO find way to supply this
// Override the webstore variables to hard coded defaults
webKeyStoreName := tls.IntegrationDefaultLabel + ".p12"
// Check keystore exists
ks := filepath.Join(keyStoreDir, webKeyStoreName)
_, err := os.Stat(ks)
// Now we know if the file exists let's check whether we should have it or not.
// Check if we're being told to generate the certificate
genHostName := os.Getenv("MQ_GENERATE_CERTIFICATE_HOSTNAME")
if genHostName != "" {
// We've got to generate the certificate with the hostname given
if err == nil {
log.Printf("Replacing existing keystore %s - generating new certificate", ks)
}
// Keystore doesn't exist so create it and populate a certificate
newKS := keystore.NewPKCS12KeyStore(ks, p12TrustStore.Password)
err = newKS.Create()
if err != nil {
return "", fmt.Errorf("Failed to create keystore %s: %v", ks, err)
}
err = newKS.CreateSelfSignedCertificate("default", fmt.Sprintf("CN=%s", genHostName), genHostName)
if err != nil {
return "", fmt.Errorf("Failed to generate certificate in keystore %s with DN of 'CN=%s': %v", ks, genHostName, err)
}
} else {
// Keystore should already exist
if err != nil {
return "", fmt.Errorf("Failed to find existing keystore %s: %v", ks, err)
}
}
// Check truststore exists
_, err = os.Stat(p12TrustStore.Keystore.Filename)
if err != nil {
return "", fmt.Errorf("Failed to find existing truststore %s: %v", p12TrustStore.Keystore.Filename, err)
}
return webKeyStoreName, nil
}

View File

@@ -38,11 +38,11 @@ func logDateStamp() {
}
func logGitRepo() {
log.Printf("Image revision: %v", ImageRevision)
// log.Printf("Image revision: %v", ImageRevision)
}
func logGitCommit() {
log.Printf("Image source: %v", ImageSource)
// log.Printf("Image source: %v", ImageSource)
}
func logImageTag() {

View File

@@ -1,5 +1,3 @@
// +build mqdev
/*
© Copyright IBM Corporation 2018, 2019
@@ -19,31 +17,44 @@ package main
import (
"fmt"
"io"
"os"
"os/exec"
"os/user"
"path/filepath"
"strconv"
"strings"
"syscall"
"github.com/ibm-messaging/mq-container/internal/command"
"github.com/ibm-messaging/mq-container/internal/copy"
"github.com/ibm-messaging/mq-container/internal/mqtemplate"
"github.com/ibm-messaging/mq-container/internal/tls"
)
func startWebServer() error {
func startWebServer(keystore, keystorepw string) error {
_, err := os.Stat("/opt/mqm/bin/strmqweb")
if err != nil && os.IsNotExist(err) {
log.Debug("Skipping web server, because it's not installed")
return nil
}
log.Println("Starting web server")
// #nosec G204 - command is fixed, no injection vector
cmd := exec.Command("strmqweb")
// Set a default app password for the web server, if one isn't already set
_, set := os.LookupEnv("MQ_APP_PASSWORD")
if !set {
// Take all current environment variables, and add the app password
cmd.Env = append(os.Environ(), "MQ_APP_PASSWORD=passw0rd")
} else {
cmd.Env = os.Environ()
}
// TLS enabled
if keystore != "" {
cmd.Env = append(cmd.Env, "AMQ_WEBKEYSTORE="+keystore)
cmd.Env = append(cmd.Env, "AMQ_WEBKEYSTOREPW="+keystorepw)
}
uid, gid, err := command.LookupMQM()
if err != nil {
return err
@@ -70,45 +81,82 @@ func startWebServer() error {
return nil
}
// CopyFile copies the specified file
func CopyFile(src, dest string) error {
log.Debugf("Copying file %v to %v", src, dest)
in, err := os.Open(src)
if err != nil {
return err
func configureSSO(p12TrustStore tls.KeyStoreData) (string, error) {
// Ensure all required environment variables are set for SSO
requiredEnvVars := []string{
"MQ_WEB_ADMIN_USERS",
"MQ_OIDC_CLIENT_ID",
"MQ_OIDC_CLIENT_SECRET",
"MQ_OIDC_UNIQUE_USER_IDENTIFIER",
"MQ_OIDC_AUTHORIZATION_ENDPOINT",
"MQ_OIDC_TOKEN_ENDPOINT",
"MQ_OIDC_JWK_ENDPOINT",
"MQ_OIDC_ISSUER_IDENTIFIER",
}
defer in.Close()
out, err := os.OpenFile(dest, os.O_CREATE|os.O_WRONLY, 0770)
defer out.Close()
_, err = io.Copy(out, in)
if err != nil {
return err
for _, envVar := range requiredEnvVars {
if len(os.Getenv(envVar)) == 0 {
return "", fmt.Errorf("%v must be set when MQ_BETA_ENABLE_SSO=true", envVar)
}
}
err = out.Close()
return err
}
func configureWebServer() error {
_, err := os.Stat("/opt/mqm/bin/strmqweb")
// Check mqweb directory exists
const mqwebDir string = "/etc/mqm/web/installations/Installation1/servers/mqweb"
_, err := os.Stat(mqwebDir)
if err != nil {
if os.IsNotExist(err) {
return nil
return "", nil
}
return err
return "", err
}
// Process SSO template for generating file mqwebuser.xml
adminUsers := strings.Split(os.Getenv("MQ_WEB_ADMIN_USERS"), "\n")
err = mqtemplate.ProcessTemplateFile(mqwebDir+"/mqwebuser.xml.tpl", mqwebDir+"/mqwebuser.xml", map[string][]string{"AdminUser": adminUsers}, log)
if err != nil {
return "", err
}
// Configure SSO TLS
return configureSSOTLS(p12TrustStore)
}
func configureWebServer(keyLabel string, p12Trust tls.KeyStoreData) (string, error) {
var keystore string
// Configure TLS for Web Console first if we have a certificate to use
err := configureWebTLS(keyLabel)
if err != nil {
return keystore, err
}
if keyLabel != "" {
keystore = keyLabel + ".p12"
}
// Configure Single-Sign-On for the web server (if enabled)
enableSSO := os.Getenv("MQ_BETA_ENABLE_SSO")
if enableSSO == "true" || enableSSO == "1" {
keystore, err = configureSSO(p12Trust)
if err != nil {
return keystore, err
}
}
_, err = os.Stat("/opt/mqm/bin/strmqweb")
if err != nil {
if os.IsNotExist(err) {
return keystore, nil
}
return keystore, err
}
const webConfigDir string = "/etc/mqm/web"
_, err = os.Stat(webConfigDir)
if err != nil {
if os.IsNotExist(err) {
return nil
return keystore, nil
}
return err
return keystore, err
}
uid, gid, err := command.LookupMQM()
if err != nil {
return err
return keystore, err
}
const prefix string = "/etc/mqm/web"
err = filepath.Walk(prefix, func(from string, info os.FileInfo, err error) error {
@@ -127,6 +175,7 @@ func configureWebServer() error {
}
if info.IsDir() {
if !exists {
// #nosec G301 - write group permissions are required
err := os.MkdirAll(to, 0770)
if err != nil {
return err
@@ -139,7 +188,7 @@ func configureWebServer() error {
return err
}
}
err := CopyFile(from, to)
err := copy.CopyFile(from, to)
if err != nil {
log.Error(err)
return err
@@ -151,5 +200,5 @@ func configureWebServer() error {
}
return nil
})
return err
return keystore, err
}

View File

@@ -2,40 +2,22 @@
## Prerequisites
### Prerequisites for building an Ubuntu image
If you want to build a container image with Ubuntu Linux as the base OS, then you need to have the following tools installed:
You need to have the following tools installed:
* [Docker](https://www.docker.com/) V17.06.1 or later
* [GNU make](https://www.gnu.org/software/make/)
If you are working in the Windows Subsystem for Linux, follow [this guide by Microsoft to set up Docker](https://blogs.msdn.microsoft.com/commandline/2017/12/08/cross-post-wsl-interoperability-with-docker/) first.
### Prerequisites for building a Red Hat Enterprise Linux image
If you want to build a container image with Red Hat Enterprise Linux as the base OS, then you need to use a host server with Red Hat Enterprise Linux. You must also have the following tools installed:
* [`buildah`](https://buildah.io) (available in `rhel-7-server-extras`)
* [`podman`](https://podman.io) (available in `rhel-7-server-extras`)
In addition, you need the following commonly installed tools:
* `bash`
* `coreutils`
* `findutils`
* `make`
* `sed`
* `shadow-utils`
* `tar`
## Building a production image
This procedure works for building the MQ Continuous Delivery release, on `x86_64`, `ppc64le` and `s390x` architectures.
This procedure works for building the MQ Continuous Delivery release, on `amd64`, `ppc64le` and `s390x` architectures.
1. Create a `downloads` directory in the root of this repository
2. Download MQ from [IBM Passport Advantage](https://www.ibm.com/software/passportadvantage/) or [IBM Fix Central](https://www.ibm.com/support/fixcentral), and place the downloaded file (for example, `IBM_MQ_9.1.1_UBUNTU_X86-64.tar.gz` for MQ V9.1.1 for Ubuntu on x86_64 architecture) in the `downloads` directory
2. Download MQ from [IBM Passport Advantage](https://www.ibm.com/software/passportadvantage/) or [IBM Fix Central](https://www.ibm.com/support/fixcentral), and place the downloaded file (for example, `IBM_MQ_9.1.3_LINUX_X86-64.tar.gz`) in the `downloads` directory
3. Run `make build-advancedserver`
> **Warning**: Note that MQ offers two different sets of packaging on Linux: one is called "MQ for Linux" and contains RPM files for installing on Red Hat Enterprise Linux and SUSE Linux Enterprise Server. The other package is called "MQ for Ubuntu", and contains DEB files for installing on Ubuntu.
On a Red Hat Enterprise Linux host, the command `make build-advancedserver` will build a container image using Red Hat Enterprise Linux as the base. On all other hosts, the base image will be Ubuntu.
> **Warning**: Note that MQ offers two different sets of packaging on Linux: one is called "MQ for Linux" and contains RPM files for installing on Red Hat Enterprise Linux and SUSE Linux Enterprise Server. The MQ container build uses a Red Hat Universal Base Image, so you need the "MQ for Linux" RPM files.
You can build a different version of MQ by setting the `MQ_VERSION` environment variable, for example:
@@ -50,10 +32,15 @@ MQ_ARCHIVE=mq-1.2.3.4.tar.gz MQ_VERSION=1.2.3.4 make build-advancedserver
```
## Building a developer image
Run `make build-devserver`, which will download the latest version of MQ Advanced for Developers from IBM developerWorks. This is currently only available on the `x86_64` architecture. On a Red Hat Enterprise Linux host, this command will build a container image using Red Hat Enterprise Linux as the base. On all other hosts, the base image will be Ubuntu.
Run `make build-devserver`, which will download the latest version of MQ Advanced for Developers from IBM developerWorks. This is currently only available on the `amd64` architecture.
You can use the environment variable `MQ_ARCHIVE_DEV` to specify an alternative local file to install from (which must be in the `downloads` directory).
## Building from a Red Hat Enterprise Linux host
Red Hat Enterprise Linux (RHEL) offers a suite of container tools, including Buildah for building container images, and Podman for running containers. Buildah can accept input described in a [Dockerfile](https://docs.docker.com/engine/reference/builder/). This MQ sample uses a multi-stage build, which requires a recent version of Podman, which is not yet available in Red Hat Enterprise Linux V7. Therefore, if you are on a RHEL host, then the `build-devserver` and `build-advancedserver` targets are run using a more recent version of Buildah from inside a container.
The containerized build process on a RHEL host will write an OCI compliant archive file to `/tmp/mq-buildah`. If a version of Docker is installed on the host, it will also push the image into Docker's internal image registry.
## Installed components
This image includes the core MQ server, Java, language packs, and GSKit. This can be configured by setting the `MQ_PACKAGES` argument to `make`. For the Ubuntu-based image, you can also directly set a [Docker build argument](https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables-build-arg).
This image includes the core MQ server, Java, language packs, GSKit, and web server. This can be configured by setting the `MQ_PACKAGES` argument to `make`.

View File

@@ -9,8 +9,8 @@ The MQ Developer Defaults supports some customization options, these are all con
* **MQ_DEV** - Set this to `false` to stop the default objects being created.
* **MQ_ADMIN_PASSWORD** - Changes the password of the `admin` user. Must be at least 8 characters long.
* **MQ_APP_PASSWORD** - Changes the password of the app user. If set, this will cause the `DEV.APP.SVRCONN` channel to become secured and only allow connections that supply a valid userid and password. Must be at least 8 characters long.
* **MQ_TLS_KEYSTORE** - Allows you to supply the location of a PKCS#12 keystore containing a single certificate which you want to use in both the web console and the queue manager. Requires `MQ_TLS_PASSPHRASE`. When enabled the channels created will be secured using the `TLS_RSA_WITH_AES_128_CBC_SHA256` CipherSpec. *Note*: you will need to make the keystore available inside your container, this can be done by mounting a volume to your container.
* **MQ_TLS_PASSPHRASE** - Passphrase for the keystore referenced in `MQ_TLS_KEYSTORE`.
* **MQ_TLS_KEYSTORE** - **DEPRECATED**. See section `Supplying TLS certificates` in [usage document](usage.md). Allows you to supply the location of a PKCS#12 keystore containing a single certificate which you want to use in both the web console and the queue manager. Requires `MQ_TLS_PASSPHRASE`. When enabled the channels created will be secured using the `TLS_RSA_WITH_AES_128_CBC_SHA256` CipherSpec. *Note*: you will need to make the keystore available inside your container, this can be done by mounting a volume to your container.
* **MQ_TLS_PASSPHRASE** - **DEPRECATED**. See section `Supplying TLS certificates` in [usage document](usage.md). Passphrase for the keystore referenced in `MQ_TLS_KEYSTORE`.
## Details of the default configuration
@@ -52,4 +52,4 @@ If you choose to accept the security warning, you will be presented with the log
If you wish to change the password for the admin user, this can be done using the `MQ_ADMIN_PASSWORD` environment variable. If you supply a PKCS#12 keystore using the `MQ_TLS_KEYSTORE` environment variable, then the web console will be configured to use the certificate inside the keystore for HTTPS operations.
If you do not wish the web console to run, you can disable it by setting the environment variable `MQ_DISABLE_WEB_CONSOLE` to `true`.
If you do not wish the web console to run, you can disable it by setting the environment variable `MQ_ENABLE_EMBEDDED_WEB_SERVER` to `false`.

View File

@@ -24,6 +24,7 @@ The `runmqserver` command has the following responsibilities:
- Works as PID 1, so is responsible for [reaping zombie processes](https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/)
* Creating and starting a queue manager
* Configuring the queue manager, by running any MQSC scripts found under `/etc/mqm`
* Starts the MQ web server (if enabled)
* Starting Prometheus metrics generation for the queue manager (if enabled)
* Indicates to the `chkmqready` command that configuration is complete, and that normal readiness checking can happen. This is done by writing a file into `/run/runmqserver`
@@ -36,8 +37,6 @@ The `runmqdevserver` command is added to the MQ Advanced for Developers image on
2. Generates MQSC files to put in `/etc/mqm`, based on a template, which is updated with values based on supplied environment variables.
3. If requested, it creates TLS key stores under `/run/runmqdevserver`, and configures MQ and the web server to use them
A special version of `runmqserver` is used in the developer image, which performs extra actions like starting the web server. This is built using the `mqdev` [build constraint](https://golang.org/pkg/go/build/#hdr-Build_Constraints).
## Prometheus metrics
[Prometheus](https://prometheus.io) metrics are generated for the queue manager as follows:

View File

@@ -4,7 +4,7 @@
### User
The MQ server image is run using the "mqm" user. On the Ubuntu-based image, this uses the UID and GID of 999. On the Red Hat Enterprise Linux image, it uses the UID and GID of 888.
The MQ server image is run using the "mqm" user, with a fixed UID and GID of 888.
### Capabilities
@@ -16,7 +16,7 @@ docker run \
--env LICENSE=accept \
--env MQ_QMGR_NAME=QM1 \
--detach \
mqadvanced-server:9.1.2.0-x86_64-ubuntu-16.04
mqadvanced-server:9.1.3.0-amd64
```
The MQ Advanced for Developers image does require the "chown", "setuid", "setgid" and "audit_write" capabilities (plus "dac_override" if you're using an image based on Red Hat Enterprise Linux). This is because it uses the "sudo" command to change passwords inside the container. For example, in Docker, you could do the following:
@@ -31,9 +31,9 @@ docker run \
--env LICENSE=accept \
--env MQ_QMGR_NAME=QM1 \
--detach \
mqadvanced-server-dev:9.1.2.0-x86_64-ubuntu-16.04
mqadvanced-server-dev:9.1.3.0-amd64
```
### SELinux
The SELinux label "spc_t" (super-privileged container) is needed to run the MQ container on a host with SELinux enabled. This is due to a current limitation in how MQ data is stored on volumes, which violates the usual policy applied when using the standard "container_t" label.
The SELinux label "spc_t" (super-privileged container) is needed to run the MQ container on a host with SELinux enabled. This is due to a current limitation in how MQ data is stored on volumes, which violates the usual policy applied when using the standard "container_t" label.

View File

@@ -6,13 +6,6 @@ You need to ensure you have the following tools installed:
* [GNU make](https://www.gnu.org/software/make/)
* [Go](https://golang.org/) - only needed for running the tests
* [dep](https://github.com/golang/dep) (official Go dependency management tool) - needed to prepare for running the tests
* [Helm](https://helm.sh) - only needed for running the Kubernetes tests
### Prerequisites for testing a RedHat image
If you want to test a container image with Red Hat Enterprise Linux as the base OS, then you need to use a host server with Red Hat Enterprise Linux. You must also have the following tools installed:
* [Yum](http://yum.baseurl.org/) (available in `rhel-7-server-extras`)
* [Buildah](https://buildah.io) (available in `rhel-7-server-extras`)
## Running the tests
There are two main sets of tests:
@@ -31,7 +24,7 @@ make test-advancedserver
You can specify the image to use directly by using the `MQ_IMAGE_ADVANCEDSERVER` or `MQ_IMAGE_DEVSERVER` variables, for example:
```
MQ_IMAGE_ADVANCEDSERVER=mqadvanced-server:9.1.2.0-x86_64-ubuntu-16.04 make test-advancedserver
MQ_IMAGE_ADVANCEDSERVER=mqadvanced-server:9.1.3.0-amd64 make test-advancedserver
```
You can pass parameters to `go test` with an environment variable. For example, to run the "TestGoldenPath" test, run the following command::
@@ -40,10 +33,10 @@ You can pass parameters to `go test` with an environment variable. For example,
TEST_OPTS_DOCKER="-run TestGoldenPath" make test-advancedserver
```
You can also use the same environment variables you specified when [building](./building), for example, the following will try and test an image called `mqadvanced-server:9.1.2.0-x86_64-ubuntu-16.04`:
You can also use the same environment variables you specified when [building](./building), for example, the following will try and test an image called `mqadvanced-server:9.1.3.0-amd64`:
```
MQ_VERSION=9.1.2.0 make test-advancedserver
MQ_VERSION=9.1.3.0 make test-advancedserver
```
### Running the Docker tests with code coverage

View File

@@ -96,3 +96,22 @@ docker exec \
```
Using this technique, you can have full control over all aspects of the MQ installation. Note that if you use this technique to make changes to the filesystem, then those changes would be lost if you re-created your container unless you make those changes in volumes.
## Supplying TLS certificates
If you wish to supply TLS Certificates that the queue manager and MQ Console should use for TLS operations then you must supply a PKCS#1 or unencrypted PKCS#8 PEM files for both the certificates and private keys in the following directories:
* `/etc/mqm/pki/keys/<Label>` - for certificates with public and private keys
* `/etc/mqm/pki/trust/<index>` - for certificates with only the public key
For example, if you have an identity certificate you wish to add with the label `mykey` and 2 certificates you wish to add as trusted then you would need to add the files into the following locations where files ending in `.key` contain private keys and `.crt` contain certificates:
- `/etc/mqm/pki/keys/mykey/tls.key`
- `/etc/mqm/pki/keys/mykey/tls.crt`
- `/etc/mqm/pki/keys/mykey/ca.crt`
- `/etc/mqm/pki/trust/0/tls.crt`
- `/etc/mqm/pki/trust/1/tls.crt`
This can be achieved by either mounting the directories or files into the container when you run it or by baking the files into the correct location in the image.
If you supply multiple identity certificates then the first label alphabetically will be chosen as the certificate to be used by the MQ Console and the default certificate for the queue manager. If you wish to use a different certificate on the queue manager then you can change the certificate to use at runtime by executing the MQSC command `ALTER QMGR CERTLABL('<newlabel>')`

19
etc/mqm/15-tls.mqsc.tpl Normal file
View File

@@ -0,0 +1,19 @@
* © Copyright IBM Corporation 2019
*
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* Set the keystore location for the queue manager
ALTER QMGR SSLKEYR('{{ .SSLKeyR }}')
ALTER QMGR CERTLABL('{{ .CertificateLabel }}')

8
glide.lock generated
View File

@@ -1,5 +1,5 @@
hash: b02555ebf3957ece0ae5ecf132fa4e415a4f66a7f4c27a82d484f4fb78f56e41
updated: 2018-07-13T08:50:32.923040349+01:00
hash: 6ebd5fb1c39729378c7256da6f312e9699bff1ddff9941d3c8c1ba785e22acfd
updated: 2019-05-21T10:38:01.227081+01:00
imports:
- name: github.com/beorn7/perks
version: 3a771d992973f24aa725d07868b467d1ddfceafb
@@ -50,4 +50,8 @@ imports:
version: 1b2967e3c290b7c545b3db0deeda16e9be4f98a2
subpackages:
- unix
- name: software.sslmate.com/src/go-pkcs12
version: 6e380ad96778cc63c6ea17649a9b74224bceafe9
subpackages:
- internal/rc2
testImports: []

View File

@@ -1,4 +1,4 @@
# © Copyright IBM Corporation 2017
# © Copyright IBM Corporation 2017, 2019
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -25,4 +25,6 @@ import:
- package: github.com/ibm-messaging/mq-golang
version: 2.0.0
- package: github.com/genuinetools/amicontained
version: 0.4.0
version: 0.4.0
- package: software.sslmate.com/src/go-pkcs12
commit: 6e380ad96778cc63c6ea17649a9b74224bceafe9

View File

@@ -12,29 +12,33 @@
# See the License for the specific language governing permissions and
# limitations under the License.
FROM ubuntu:16.04
FROM registry.access.redhat.com/ubi7/ubi-minimal AS mq-explorer
# The URL to download the MQ installer from in tar.gz format
ARG MQ_URL=https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqadv/mqadv_dev911_ubuntu_x86-64.tar.gz
ARG MQ_URL="https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/messaging/mqadv/mqadv_dev912_linux_x86-64.tar.gz"
# The MQ packages to install
ARG MQ_PACKAGES
ENV MQ_PACKAGES="MQSeriesRuntime*.rpm MQSeriesJRE*.rpm MQSeriesExplorer*.rpm"
ARG MQM_UID=999
ARG MQM_UID=888
RUN export DEBIAN_FRONTEND=noninteractive \
&& apt-get update \
&& apt-get install -y \
libgtk2.0-0 \
libxtst6
RUN microdnf install -y --nodocs gtk2 libXtst \
&& microdnf clean all
ADD install-mq.sh /usr/local/bin/
# Install MQ Explorer. To avoid a "text file busy" error here, we sleep before installing.
# Need to re-instate the `/var/mqm` directory after installation, to avoid MQ
# errors with some commands (e.g. `dspmqver`)
RUN chmod u+x /usr/local/bin/install-mq.sh \
&& install-mq.sh
&& sleep 1 \
&& install-mq.sh $MQM_UID \
&& rm -rf /var/mqm \
&& /opt/mqm/bin/crtmqdir -f -s
ENV LANG=en_US.UTF-8
# Run as mqm (999)
USER 999
# Run as mqm
USER $MQM_UID
ENTRYPOINT ["MQExplorer"]

View File

@@ -1,33 +0,0 @@
# © Copyright IBM Corporation 2018, 2019
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
ARG BASE_IMAGE=mq-sdk:9.1.1.0-x86_64-ubuntu-16.04
FROM $BASE_IMAGE
COPY incubating/mq-golang-sdk/install-golang.sh /usr/local/bin
ENV GO_VERSION=1.10
ENV PATH="${PATH}:/usr/lib/go-${GO_VERSION}/bin:/go/bin:/usr/local/go/bin" \
CGO_CFLAGS="-I/opt/mqm/inc/" \
CGO_LDFLAGS_ALLOW="-Wl,-rpath.*" \
GOPATH="/go"
# Install the Go compiler and Git
RUN chmod +x /usr/local/bin/install-golang.sh \
&& sleep 1 \
&& install-golang.sh
WORKDIR $GOPATH

View File

@@ -1,5 +0,0 @@
# IBM MQ Software Developer Kit (SDK) with Go
This image contains the MQ SDK, Git, the Go compiler, and the `build-essential` package (which includes GNU C and C++ compilers plus other essential tools like `make`).
This image doesn't contain any Go code for MQ. You can add a CGO wrapper for the MQ C client, for example [mq-golang](https://github.com/ibm-messaging/mq-golang), via your vendor directory, or directly using `go get`.

View File

@@ -12,19 +12,19 @@
# See the License for the specific language governing permissions and
# limitations under the License.
ARG BASE_IMAGE=ubuntu:16.04
FROM $BASE_IMAGE
FROM registry.access.redhat.com/rhscl/devtoolset-7-toolchain-rhel7 AS mq-sdk
#FROM docker.io/centos/devtoolset-7-toolchain-centos7 AS mq-sdk
# The URL to download the MQ installer from in tar.gz format
# This assumes an archive containing the MQ Debian (.deb) install packages
ARG MQ_URL
# The packages to install in install-mq.sh
ARG MQ_PACKAGES
ENV MQ_PACKAGES="MQSeriesRuntime-*.rpm MQSeriesSDK-*.rpm MQSeriesSamples*.rpm"
ARG MQM_UID=999
ENV MQM_UID=888
USER 0
COPY install-mq.sh /usr/local/bin/
# Install MQ. To avoid a "text file busy" error here, we sleep before installing.
@@ -35,3 +35,4 @@ RUN chmod u+x /usr/local/bin/install-mq.sh \
&& install-mq.sh $MQM_UID \
&& rm -rf /var/mqm \
&& /opt/mqm/bin/crtmqdir -f -s
USER 1001

View File

@@ -14,6 +14,7 @@
* limitations under the License.
STOP LISTENER('SYSTEM.LISTENER.TCP.1') IGNSTATE(YES)
ALTER LISTENER('SYSTEM.LISTENER.TCP.1') TRPTYPE(TCP) CONTROL(MANUAL)
* Developer queues
DEFINE QLOCAL('DEV.QUEUE.1') REPLACE

View File

@@ -1,4 +1,4 @@
* © Copyright IBM Corporation 2018
* © Copyright IBM Corporation 2018, 2019
*
*
* Licensed under the Apache License, Version 2.0 (the "License");
@@ -13,10 +13,6 @@
* See the License for the specific language governing permissions and
* limitations under the License.
* Set the keystore location for the queue manager
ALTER QMGR SSLKEYR('{{ .SSLKeyR }}')
ALTER QMGR CERTLABL('{{ .CertificateLabel }}')
* Set the cipherspec for dev channels
ALTER CHANNEL('DEV.APP.SVRCONN') CHLTYPE(SVRCONN) SSLCIPH({{ .SSLCipherSpec }}) SSLCAUTH(OPTIONAL)
ALTER CHANNEL('DEV.ADMIN.SVRCONN') CHLTYPE(SVRCONN) SSLCIPH({{ .SSLCipherSpec }}) SSLCAUTH(OPTIONAL)

View File

@@ -1,86 +0,0 @@
# © Copyright IBM Corporation 2015, 2019
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
ARG BASE_IMAGE=mqadvanced-server-dev-base:9.1.2.0-x86_64-ubuntu-16.04
ARG BUILDER_IMAGE=mq-golang-sdk:9.1.2.0-x86_64-ubuntu-16.04
###############################################################################
# Build stage to build Go code
###############################################################################
FROM $BUILDER_IMAGE as builder
ARG IMAGE_REVISION="Not specified"
ARG IMAGE_SOURCE="Not specified"
ARG IMAGE_TAG="Not specified"
WORKDIR /go/src/github.com/ibm-messaging/mq-container/
COPY cmd/ ./cmd
COPY internal/ ./internal
COPY vendor/ ./vendor
# Re-build runmqserver, with code tagged with 'mqdev' enabled
RUN go build -ldflags "-X \"main.ImageCreated=$(date --iso-8601=seconds)\" -X \"main.ImageRevision=$IMAGE_REVISION\" -X \"main.ImageSource=$IMAGE_SOURCE\" -X \"main.ImageTag=$IMAGE_TAG\"" --tags 'mqdev' ./cmd/runmqserver
RUN go build ./cmd/runmqdevserver/
# Run all unit tests
RUN go test -v ./cmd/runmqdevserver/...
###############################################################################
# Main build stage
###############################################################################
FROM $BASE_IMAGE
# Enable MQ developer default configuration
ENV MQ_DEV=true
# Default administrator password
ENV MQ_ADMIN_PASSWORD=passw0rd
ARG MQM_UID=999
USER root
COPY incubating/mqadvanced-server-dev/install-extra-packages.sh /usr/local/bin/
RUN chmod u+x /usr/local/bin/install-extra-packages.sh \
&& sleep 1 \
&& install-extra-packages.sh
# WARNING: This is what allows the mqm user to change the password of any other user
# It's used by runmqdevserver to change the admin/app passwords.
RUN echo "mqm ALL = NOPASSWD: /usr/sbin/chpasswd" > /etc/sudoers.d/mq-dev-config
## Add admin and app users, and set a default password for admin
RUN useradd admin -G mqm \
&& groupadd mqclient \
&& useradd app -G mqclient \
&& echo admin:$MQ_ADMIN_PASSWORD | chpasswd
# Create a directory for runtime data from runmqserver
RUN mkdir -p /run/runmqdevserver \
&& chown mqm:mqm /run/runmqdevserver
COPY --from=builder /go/src/github.com/ibm-messaging/mq-container/runmqserver /usr/local/bin/
COPY --from=builder /go/src/github.com/ibm-messaging/mq-container/runmqdevserver /usr/local/bin/
# Copy template files
COPY incubating/mqadvanced-server-dev/*.tpl /etc/mqm/
# Copy web XML files for default developer configuration
COPY incubating/mqadvanced-server-dev/web /etc/mqm/web
RUN chown -R mqm:mqm /etc/mqm/* \
&& chmod +x /usr/local/bin/runmq* \
&& install --directory --mode 0775 --owner mqm --group root /run/runmqdevserver
EXPOSE 9443
USER $MQM_UID
ENTRYPOINT ["runmqdevserver"]

View File

@@ -1,17 +0,0 @@
#!/bin/bash
# Change admin password
if [ -n "${MQ_ADMIN_PASSWORD}" ]; then
echo admin:${MQ_ADMIN_PASSWORD} | chpasswd
fi
# Change app password
if [ -n "${MQ_APP_PASSWORD}" ]; then
echo app:${MQ_APP_PASSWORD} | chpasswd
fi
# Delete the MQSC with developer defaults, if requested
if [ "${MQ_DEV}" != "true" ]; then
rm -f /etc/mqm/dev.mqsc
fi
exec runmqserver

View File

@@ -15,7 +15,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.
test -f /usr/bin/yum && RHEL=true || RHEL=false
test -f /usr/bin/yum && YUM=true || YUM=false
test -f /usr/bin/microdnf && MICRODNF=true || MICRODNF=false
test -f /usr/bin/apt-get && UBUNTU=true || UBUNTU=false
if ($UBUNTU); then
@@ -25,8 +26,13 @@ if ($UBUNTU); then
rm -rf /var/lib/apt/lists/*
fi
if ($RHEL); then
if ($YUM); then
yum -y install sudo
yum -y clean all
rm -rf /var/cache/yum/*
fi
if ($MICRODNF); then
microdnf install --nodocs sudo
microdnf clean all
fi

View File

@@ -1,7 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<server>
<keyStore id="MQWebKeyStore" location="/run/runmqdevserver/tls/key.jks" type="JKS" password="${env.MQ_TLS_PASSPHRASE}"/>
<keyStore id="MQWebTrustStore" location="/run/runmqdevserver/tls/trust.jks" type="JKS" password="${env.MQ_TLS_PASSPHRASE}"/>
<ssl id="thisSSLConfig" clientAuthenticationSupported="true" keyStoreRef="MQWebKeyStore" trustStoreRef="MQWebTrustStore" sslProtocol="TLSv1.2" serverKeyAlias="devcert"/>
<sslDefault sslRef="thisSSLConfig"/>
</server>

View File

@@ -1,6 +1,6 @@
#!/bin/bash
# -*- mode: sh -*-
# © Copyright IBM Corporation 2018
# © Copyright IBM Corporation 2015, 2019
#
#
# Licensed under the Apache License, Version 2.0 (the "License");
@@ -18,7 +18,9 @@
# Fail on any non-zero return code
set -ex
test -f /usr/bin/yum && RHEL=true || RHEL=false
test -f /usr/bin/yum && YUM=true || YUM=false
test -f /usr/bin/microdnf && MICRODNF=true || MICRODNF=false
test -f /usr/bin/rpm && RPM=true || RPM=false
test -f /usr/bin/apt-get && UBUNTU=true || UBUNTU=false
if ($UBUNTU); then
@@ -37,37 +39,43 @@ if ($UBUNTU); then
# This ensures no unsupported code gets installed, and makes the build faster
echo "deb ${APT_URL} ${UBUNTU_CODENAME} main restricted" > /etc/apt/sources.list
echo "deb ${APT_URL} ${UBUNTU_CODENAME}-updates main restricted" >> /etc/apt/sources.list
echo "deb ${APT_URL} ${UBUNTU_CODENAME}-backports main restricted universe" >> /etc/apt/sources.list;
echo "deb ${APT_URL} ${UBUNTU_CODENAME}-security main restricted" >> /etc/apt/sources.list
apt-get update
apt-get install -y --no-install-recommends \
golang-${GO_VERSION} \
git \
ca-certificates
fi
if ($RHEL); then
# Install additional packages required by MQ, this install process and the runtime scripts
yum -y install \
git \
apt-get update
apt-get install -y --no-install-recommends \
bash \
bc \
ca-certificates \
coreutils \
curl \
debianutils \
file \
findutils \
gawk \
grep \
libc-bin \
mount \
passwd \
procps \
sed \
tar \
gcc
cd /tmp
curl -LO https://storage.googleapis.com/golang/go${GO_VERSION}.linux-amd64.tar.gz
tar -C /usr/local -xzf go${GO_VERSION}.linux-amd64.tar.gz
util-linux
fi
# Remove any orphaned packages
$UBUNTU && apt-get autoremove -y
if ($RPM); then
EXTRA_RPMS="bash bc ca-certificates coreutils file findutils gawk glibc-common grep passwd procps-ng sed shadow-utils tar util-linux which"
# Install additional packages required by MQ, this install process and the runtime scripts
$YUM && yum -y install --setopt install_weak_deps=false ${EXTRA_RPMS}
$MICRODNF && microdnf install --nodocs ${EXTRA_RPMS}
fi
# Apply any bug fixes not included in base Ubuntu or MQ image.
# Don't upgrade everything based on Docker best practices https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/#run
$UBUNTU && apt-get install -y libapparmor1 libsystemd0 systemd systemd-sysv libudev1 perl-base --only-upgrade
# End of bug fixes
# Clean up cached files
$UBUNTU && rm -rf /var/lib/apt/lists/*
$RHEL && yum -y clean all
$RHEL && rm -rf /var/cache/yum/*
# Make the GOLANG directories
mkdir -p $GOPATH/src $GOPATH/bin
chmod -R 777 $GOPATH
$YUM && yum -y clean all
$YUM && rm -rf /var/cache/yum/*
$MICRODNF && microdnf clean all

View File

@@ -18,90 +18,19 @@
# Fail on any non-zero return code
set -ex
mqm_uid=${1:-999}
mqm_uid=${1:-888}
test -f /usr/bin/yum && RHEL=true || RHEL=false
test -f /usr/bin/yum && YUM=true || YUM=false
test -f /usr/bin/microdnf && MICRODNF=true || MICRODNF=false
test -f /usr/bin/rpm && RPM=true || RPM=false
test -f /usr/bin/apt-get && UBUNTU=true || UBUNTU=false
# If MQ_PACKAGES isn't specifically set, then choose a valid set of defaults
if [ -z "$MQ_PACKAGES" ]; then
$UBUNTU && MQ_PACKAGES="ibmmq-server ibmmq-java ibmmq-jre ibmmq-gskit ibmmq-msg-.* ibmmq-samples ibmmq-ams"
$RHEL && MQ_PACKAGES="MQSeriesRuntime-*.rpm MQSeriesServer-*.rpm MQSeriesJava*.rpm MQSeriesJRE*.rpm MQSeriesGSKit*.rpm MQSeriesMsg*.rpm MQSeriesSamples*.rpm MQSeriesAMS-*.rpm"
fi
if ($UBUNTU); then
export DEBIAN_FRONTEND=noninteractive
# Use a reduced set of apt repositories.
# This ensures no unsupported code gets installed, and makes the build faster
source /etc/os-release
# Figure out the correct apt URL based on the CPU architecture
CPU_ARCH=$(uname -p)
if [ ${CPU_ARCH} == "x86_64" ]; then
APT_URL="http://archive.ubuntu.com/ubuntu/"
else
APT_URL="http://ports.ubuntu.com/ubuntu-ports/"
fi
# Use a reduced set of apt repositories.
# This ensures no unsupported code gets installed, and makes the build faster
echo "deb ${APT_URL} ${UBUNTU_CODENAME} main restricted" > /etc/apt/sources.list
echo "deb ${APT_URL} ${UBUNTU_CODENAME}-updates main restricted" >> /etc/apt/sources.list
echo "deb ${APT_URL} ${UBUNTU_CODENAME}-security main restricted" >> /etc/apt/sources.list
# Install additional packages required by MQ, this install process and the runtime scripts
apt-get update
apt-get install -y --no-install-recommends \
bash \
bc \
ca-certificates \
coreutils \
curl \
debianutils \
file \
findutils \
gawk \
grep \
libc-bin \
mount \
passwd \
procps \
sed \
tar \
util-linux
fi
# Install additional packages required by MQ, this install process and the runtime scripts
$RHEL && yum -y install \
bash \
bc \
ca-certificates \
coreutils \
curl \
file \
findutils \
gawk \
glibc-common \
grep \
passwd \
procps-ng \
sed \
tar \
util-linux
# Download and extract the MQ installation files
DIR_EXTRACT=/tmp/mq
mkdir -p ${DIR_EXTRACT}
cd ${DIR_EXTRACT}
curl -LO $MQ_URL
tar -zxvf ./*.tar.gz
# Remove packages only needed by this script
$UBUNTU && apt-get purge -y \
ca-certificates \
curl
# Note: ca-certificates and curl are installed by default in RHEL
# Remove any orphaned packages
$UBUNTU && apt-get autoremove -y
tar -zxf ./*.tar.gz
# Recommended: Create the mqm user ID with a fixed UID and group, so that the file permissions work between different images
groupadd --system --gid ${mqm_uid} mqm
@@ -109,7 +38,7 @@ useradd --system --uid ${mqm_uid} --gid mqm --groups 0 mqm
# Find directory containing .deb files
$UBUNTU && DIR_DEB=$(find ${DIR_EXTRACT} -name "*.deb" -printf "%h\n" | sort -u | head -1)
$RHEL && DIR_RPM=$(find ${DIR_EXTRACT} -name "*.rpm" -printf "%h\n" | sort -u | head -1)
$RPM && DIR_RPM=$(find ${DIR_EXTRACT} -name "*.rpm" -printf "%h\n" | sort -u | head -1)
# Find location of mqlicense.sh
MQLICENSE=$(find ${DIR_EXTRACT} -name "mqlicense.sh")
@@ -121,10 +50,11 @@ $UBUNTU && echo "deb [trusted=yes] file:${DIR_DEB} ./" > /etc/apt/sources.list.d
$UBUNTU && apt-get update
$UBUNTU && apt-get install -y $MQ_PACKAGES
$RHEL && cd $DIR_RPM && rpm -ivh $MQ_PACKAGES
$RPM && cd $DIR_RPM && rpm -ivh $MQ_PACKAGES
# Remove 32-bit libraries from 64-bit container
find /opt/mqm /var/mqm -type f -exec file {} \; | awk -F: '/ELF 32-bit/{print $1}' | xargs --no-run-if-empty rm -f
# The "file" utility isn't installed by default in UBI, so only try this if it's installed
which file && find /opt/mqm /var/mqm -type f -exec file {} \; | awk -F: '/ELF 32-bit/{print $1}' | xargs --no-run-if-empty rm -f
# Remove tar.gz files unpacked by RPM postinst scripts
find /opt/mqm -name '*.tar.gz' -delete
@@ -136,16 +66,6 @@ find /opt/mqm -name '*.tar.gz' -delete
$UBUNTU && rm -f /etc/apt/sources.list.d/IBM_MQ.list
rm -rf ${DIR_EXTRACT}
# Apply any bug fixes not included in base Ubuntu or MQ image.
# Don't upgrade everything based on Docker best practices https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/#run
$UBUNTU && apt-get install -y libapparmor1 libsystemd0 systemd systemd-sysv libudev1 perl-base --only-upgrade
# End of bug fixes
# Clean up cached files
$UBUNTU && rm -rf /var/lib/apt/lists/*
$RHEL && yum -y clean all
$RHEL && rm -rf /var/cache/yum/*
# Optional: Update the command prompt with the MQ version
$UBUNTU && echo "mq:$(dspmqver -b -f 2)" > /etc/debian_chroot
@@ -156,17 +76,34 @@ rm -rf /var/mqm
install --directory --mode 0775 --owner mqm --group root /mnt
install --directory --mode 0775 --owner mqm --group root /mnt/mqm
install --directory --mode 0775 --owner mqm --group root /mnt/mqm/data
install --directory --mode 0775 --owner mqm --group root /mnt/mqm-log
install --directory --mode 0775 --owner mqm --group root /mnt/mqm-log/log
install --directory --mode 0775 --owner mqm --group root /mnt/mqm-data
install --directory --mode 0775 --owner mqm --group root /mnt/mqm-data/qmgrs
# Create the directory for MQ configuration files
install --directory --mode 0775 --owner mqm --group root /etc/mqm
# Create the directory for MQ runtime files
install --directory --mode 0775 --owner mqm --group root /run/mqm
# Create a symlink for /var/mqm -> /mnt/mqm/data
ln -s /mnt/mqm/data /var/mqm
# Optional: Ensure any passwords expire in a timely manner
sed -i 's/PASS_MAX_DAYS\t99999/PASS_MAX_DAYS\t90/' /etc/login.defs
sed -i 's/PASS_MIN_DAYS\t0/PASS_MIN_DAYS\t1/' /etc/login.defs
sed -i 's/PASS_MIN_LEN\t5/PASS_MIN_LEN\t8/' /etc/login.defs
sed -i 's/# minlen = 9/minlen = 8/' /etc/security/pwquality.conf
$UBUNTU && PAM_FILE=/etc/pam.d/common-password
$RHEL && PAM_FILE=/etc/pam.d/password-auth
$RPM && PAM_FILE=/etc/pam.d/password-auth
sed -i 's/password\t\[success=1 default=ignore\]\tpam_unix\.so obscure sha512/password\t[success=1 default=ignore]\tpam_unix.so obscure sha512 minlen=8/' $PAM_FILE
# List all the installed packages, for the build log
$RPM && rpm -q --all || true
$UBUNTU && dpkg --list || true
# Copy MQ Licenses into the correct location
mkdir -p /licenses
cp /opt/mqm/licenses/*.txt /licenses/

View File

@@ -13,7 +13,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package runtime
package containerruntime
import (
"fmt"
@@ -107,3 +107,22 @@ func GetKernelVersion() (string, error) {
func GetMaxFileHandles() (string, error) {
return readProc("/proc/sys/fs/file-max")
}
// SupportedFilesystem returns true if the supplied filesystem type is supported for MQ data
func SupportedFilesystem(fsType string) bool {
switch fsType {
case "aufs", "overlayfs", "tmpfs":
return false
default:
return true
}
}
// ValidMultiInstanceFilesystem returns true if the supplied filesystem type is valid for a multi-instance queue manager
func ValidMultiInstanceFilesystem(fsType string) bool {
if !SupportedFilesystem(fsType) {
return false
}
// TODO : check for non-shared filesystems & shared filesystems which are known not to work
return true
}

View File

@@ -15,7 +15,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package runtime
package containerruntime
import (
"golang.org/x/sys/unix"
@@ -113,13 +113,3 @@ func GetFilesystem(path string) (string, error) {
}
return t, nil
}
// SupportedFilesystem returns true if the supplied filesystem type is supported for MQ data
func SupportedFilesystem(fsType string) bool {
switch fsType {
case "aufs", "overlayfs", "tmpfs":
return false
default:
return true
}
}

View File

@@ -15,7 +15,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package runtime
package containerruntime
// Dummy version of this function, only for non-Linux systems.
// Having this allows unit tests to be run on other platforms (e.g. macOS)

View File

@@ -13,18 +13,21 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
package containerruntimelogger
import (
"fmt"
"os"
"runtime"
"strings"
containerruntime "github.com/ibm-messaging/mq-container/internal/containerruntime"
"github.com/ibm-messaging/mq-container/internal/containerruntime"
"github.com/ibm-messaging/mq-container/internal/logger"
"github.com/ibm-messaging/mq-container/internal/user"
)
func logContainerDetails() error {
// LogContainerDetails logs details about the container runtime
func LogContainerDetails(log *logger.Logger) error {
if runtime.GOOS != "linux" {
return fmt.Errorf("Unsupported platform: %v", runtime.GOOS)
}
@@ -82,5 +85,19 @@ func logContainerDetails() error {
}
}
}
// For a multi-instance queue manager - check all required mounts exist & validate filesystem type
if os.Getenv("MQ_MULTI_INSTANCE") == "true" {
log.Println("Multi-instance queue manager: enabled")
reqMounts := []string{"/mnt/mqm", "/mnt/mqm-log", "/mnt/mqm-data"}
for _, mountPoint := range reqMounts {
if fsType, ok := m[mountPoint]; ok {
if !containerruntime.ValidMultiInstanceFilesystem(fsType) {
return fmt.Errorf("%v uses filesystem type '%v' which is invalid for a multi-instance queue manager", mountPoint, fsType)
}
} else {
return fmt.Errorf("Missing required mount '%v' for a multi-instance queue manager", mountPoint)
}
}
}
return nil
}

58
internal/copy/copy.go Normal file
View File

@@ -0,0 +1,58 @@
/*
© Copyright IBM Corporation 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package copy
import (
"fmt"
"io"
"os"
"github.com/ibm-messaging/mq-container/internal/filecheck"
)
func CopyFileMode(src, dest string, perm os.FileMode) error {
err := filecheck.CheckFileSource(src)
if err != nil {
return fmt.Errorf("failed to open %s for copy: %v", src, err)
}
// #nosec G304 - filename variable 'src' is checked above to ensure it is valid
in, err := os.Open(src)
if err != nil {
return fmt.Errorf("failed to open %s for copy: %v", src, err)
}
defer in.Close()
out, err := os.OpenFile(dest, os.O_CREATE|os.O_WRONLY, perm)
if err != nil {
return fmt.Errorf("failed to open %s for copy: %v", dest, err)
}
defer out.Close()
_, err = io.Copy(out, in)
if err != nil {
return err
}
err = out.Close()
return err
}
// CopyFile copies the specified file
func CopyFile(src, dest string) error {
return CopyFileMode(src, dest, 0770)
}

View File

@@ -1,7 +1,5 @@
// +build !mqdev
/*
© Copyright IBM Corporation 2018
© Copyright IBM Corporation 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -15,8 +13,25 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
func postInit(name string) error {
package filecheck
import (
"fmt"
"path/filepath"
"strings"
)
// CheckFileSource checks the filename is valid
func CheckFileSource(fileName string) error {
absFile, _ := filepath.Abs(fileName)
prefixes := []string{"bin", "boot", "dev", "lib", "lib64", "proc", "sbin", "sys"}
for _, prefix := range prefixes {
if strings.HasPrefix(absFile, filepath.Join("/", prefix)) {
return fmt.Errorf("Filename resolves to invalid path '%v'", absFile)
}
}
return nil
}

View File

@@ -0,0 +1,40 @@
/*
© Copyright IBM Corporation 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package filecheck
import (
"testing"
)
func TestCheckFileSource(t *testing.T) {
invalidFilenames := []string{"/bin", "/boot", "/dev", "/lib", "/lib64", "/proc", "/sbin", "/sys", "/bin/myfile", "/boot/mydir/myfile", "/var/../dev", "/var/../lib/myfile"}
for _, invalidFilename := range invalidFilenames {
err := CheckFileSource(invalidFilename)
if err == nil {
t.Errorf("Expected to receive an error for filename '%v'", invalidFilename)
}
}
validFilenames := []string{"/var", "/mydir/dev", "/mydir/dev/myfile"}
for _, validFilename := range validFilenames {
err := CheckFileSource(validFilename)
if err != nil {
t.Errorf("Unexpected error received: %v", err)
}
}
}

View File

@@ -1,5 +1,5 @@
/*
© Copyright IBM Corporation 2018
© Copyright IBM Corporation 2018, 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -13,12 +13,15 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
// Package keystore contains code to create and update keystores
package keystore
import (
"bufio"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
@@ -53,13 +56,22 @@ func NewCMSKeyStore(filename, password string) *KeyStore {
}
}
// NewPKCS12KeyStore creates a new PKCS12 Key Store, managed by the runmqakm command
func NewPKCS12KeyStore(filename, password string) *KeyStore {
return &KeyStore{
Filename: filename,
Password: password,
keyStoreType: "p12",
command: "/opt/mqm/bin/runmqakm",
}
}
// Create a key store, if it doesn't already exist
func (ks *KeyStore) Create() error {
_, err := os.Stat(ks.Filename)
if err == nil {
// Keystore already exists so we should refresh it by deleting it.
extension := filepath.Ext(ks.Filename)
log.Debugf("Refreshing keystore: %v", ks.Filename)
if ks.keyStoreType == "cms" {
// Only delete these when we are refreshing the kdb keystore
stashFile := ks.Filename[0:len(ks.Filename)-len(extension)] + ".sth"
@@ -67,23 +79,19 @@ func (ks *KeyStore) Create() error {
crlFile := ks.Filename[0:len(ks.Filename)-len(extension)] + ".crl"
err = os.Remove(stashFile)
if err != nil {
log.Errorf("Error removing %s: %v", stashFile, err)
return err
}
err = os.Remove(rdbFile)
if err != nil {
log.Errorf("Error removing %s: %v", rdbFile, err)
return err
}
err = os.Remove(crlFile)
if err != nil {
log.Errorf("Error removing %s: %v", crlFile, err)
return err
}
}
err = os.Remove(ks.Filename)
if err != nil {
log.Errorf("Error removing %s: %v", ks.Filename, err)
return err
}
} else if !os.IsNotExist(err) {
@@ -99,12 +107,10 @@ func (ks *KeyStore) Create() error {
mqmUID, mqmGID, err := command.LookupMQM()
if err != nil {
log.Error(err)
return err
}
err = os.Chown(ks.Filename, mqmUID, mqmGID)
if err != nil {
log.Error(err)
return err
}
return nil
@@ -114,7 +120,6 @@ func (ks *KeyStore) Create() error {
func (ks *KeyStore) CreateStash() error {
extension := filepath.Ext(ks.Filename)
stashFile := ks.Filename[0:len(ks.Filename)-len(extension)] + ".sth"
log.Debugf("TLS stash file: %v", stashFile)
_, err := os.Stat(stashFile)
if err != nil {
if os.IsNotExist(err) {
@@ -127,12 +132,10 @@ func (ks *KeyStore) CreateStash() error {
}
mqmUID, mqmGID, err := command.LookupMQM()
if err != nil {
log.Error(err)
return err
}
err = os.Chown(stashFile, mqmUID, mqmGID)
if err != nil {
log.Error(err)
return err
}
return nil
@@ -147,6 +150,33 @@ func (ks *KeyStore) Import(inputFile, password string) error {
return nil
}
// CreateSelfSignedCertificate creates a self-signed certificate in the keystore
func (ks *KeyStore) CreateSelfSignedCertificate(label, dn, hostname string) error {
out, _, err := command.Run(ks.command, "-cert", "-create", "-db", ks.Filename, "-pw", ks.Password, "-label", label, "-dn", dn, "-san_dnsname", hostname)
if err != nil {
return fmt.Errorf("error running \"%v -cert -create\": %v %s", ks.command, err, out)
}
return nil
}
// Add adds a CA certificate to the keystore
func (ks *KeyStore) Add(inputFile, label string) error {
out, _, err := command.Run(ks.command, "-cert", "-add", "-db", ks.Filename, "-type", ks.keyStoreType, "-pw", ks.Password, "-file", inputFile, "-label", label)
if err != nil {
return fmt.Errorf("error running \"%v -cert -add\": %v %s", ks.command, err, out)
}
return nil
}
// Add adds a CA certificate to the keystore
func (ks *KeyStore) AddNoLabel(inputFile string) error {
out, _, err := command.Run(ks.command, "-cert", "-add", "-db", ks.Filename, "-type", ks.keyStoreType, "-pw", ks.Password, "-file", inputFile)
if err != nil {
return fmt.Errorf("error running \"%v -cert -add\": %v %s", ks.command, err, out)
}
return nil
}
// GetCertificateLabels returns the labels of all certificates in the key store
func (ks *KeyStore) GetCertificateLabels() ([]string, error) {
out, _, err := command.Run(ks.command, "-cert", "-list", "-type", ks.keyStoreType, "-db", ks.Filename, "-pw", ks.Password)
@@ -171,9 +201,44 @@ func (ks *KeyStore) GetCertificateLabels() ([]string, error) {
// RenameCertificate renames the specified certificate
func (ks *KeyStore) RenameCertificate(from, to string) error {
out, _, err := command.Run(ks.command, "-cert", "-rename", "-db", ks.Filename, "-pw", ks.Password, "-label", from, "-new_label", to)
if err != nil {
return fmt.Errorf("error running \"%v -cert -rename\": %v %s", ks.command, err, out)
if ks.command == "/opt/mqm/bin/runmqakm" {
// runmqakm can't handle certs with ' in them so just use capicmd
// Overriding gosec here as this function is in an internal package and only callable by our internal functions.
// #nosec G204
cmd := exec.Command("/opt/mqm/gskit8/bin/gsk8capicmd_64", "-cert", "-rename", "-db", ks.Filename, "-pw", ks.Password, "-label", from, "-new_label", to)
cmd.Env = append(os.Environ(), "LD_LIBRARY_PATH=/opt/mqm/gskit8/lib64/:/opt/mqm/gskit8/lib")
out, _, err := command.RunCmd(cmd)
if err != nil {
return fmt.Errorf("error running \"%v -cert -rename\": %v %s", "/opt/mqm/gskit8/bin/gsk8capicmd_64", err, out)
}
} else {
out, _, err := command.Run(ks.command, "-cert", "-rename", "-db", ks.Filename, "-pw", ks.Password, "-label", from, "-new_label", to)
if err != nil {
return fmt.Errorf("error running \"%v -cert -rename\": %v %s", ks.command, err, out)
}
}
return nil
}
// ListAllCertificates Lists all certificates in the keystore
func (ks *KeyStore) ListAllCertificates() ([]string, error) {
out, _, err := command.Run(ks.command, "-cert", "-list", "-type", ks.keyStoreType, "-db", ks.Filename, "-pw", ks.Password)
if err != nil {
return nil, fmt.Errorf("error running \"%v -cert -list\": %v %s", ks.command, err, out)
}
scanner := bufio.NewScanner(strings.NewReader(out))
var labels []string
for scanner.Scan() {
s := scanner.Text()
if strings.HasPrefix(s, "-") || strings.HasPrefix(s, "*-") || strings.HasPrefix(s, "!") {
s := strings.TrimLeft(s, "-*!")
labels = append(labels, strings.TrimSpace(s))
}
}
err = scanner.Err()
if err != nil {
return nil, err
}
return labels, nil
}

View File

@@ -114,11 +114,6 @@ func (l *Logger) log(level string, msg string) {
l.mutex.Unlock()
}
// LogDirect logs a message directly to stdout
func (l *Logger) LogDirect(msg string) {
fmt.Println(msg)
}
// Debug logs a line as debug
func (l *Logger) Debug(args ...interface{}) {
if l.debug {

View File

@@ -1,5 +1,5 @@
/*
© Copyright IBM Corporation 2018
© Copyright IBM Corporation 2018, 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -24,6 +24,7 @@ import (
"time"
"github.com/ibm-messaging/mq-container/internal/logger"
"github.com/ibm-messaging/mq-container/internal/ready"
"github.com/prometheus/client_golang/prometheus"
)
@@ -39,6 +40,15 @@ var (
// GatherMetrics gathers metrics for the queue manager
func GatherMetrics(qmName string, log *logger.Logger) {
// If running in standby mode - wait until the queue manager becomes active
for {
active, _ := ready.IsRunningAsActiveQM(qmName)
if active {
break
}
time.Sleep(requestTimeout * time.Second)
}
metricsEnabled = true
err := startMetricsGathering(qmName, log)

View File

@@ -1,5 +1,5 @@
/*
© Copyright IBM Corporation 2018
© Copyright IBM Corporation 2018, 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -71,5 +71,8 @@ func GetQueueManager(name string) (*QueueManager, error) {
// GetErrorLogDirectory returns the directory holding the error logs for the
// specified queue manager
func GetErrorLogDirectory(qm *QueueManager) string {
if qm.DataPath != "" {
return filepath.Join(qm.DataPath, "errors")
}
return filepath.Join(qm.Prefix, "qmgrs", qm.Directory, "errors")
}

View File

@@ -0,0 +1,264 @@
/*
© Copyright IBM Corporation 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package mqscredact
import (
"bufio"
"io"
"regexp"
"strings"
)
/* List of sensitive MQ Parameters */
var sensitiveParameters = []string{"LDAPPWD", "PASSWORD", "SSLCRYP"}
// redactionString is what sensitive paramters will be replaced with
const redactionString = "(*********)"
func findEndOfParamterString(stringDenoter rune, r *bufio.Reader) string {
parameter := ""
for {
char, _, err := r.ReadRune()
if err != nil {
return parameter
}
parameter = parameter + string(char)
if char == stringDenoter {
break
} else if char == '\n' {
// Check if we're on a comment line
NewLineLoop:
for {
// Look at next character without moving buffer forwards
chars, err := r.Peek(1)
if err != nil {
return parameter
}
// Check if we're at the beginning of some data.
startOutput, _ := regexp.MatchString(`[^:0-9\s]`, string(chars[0]))
if startOutput {
// We are at the start, check if we're on a comment line
if chars[0] == '*' {
// found a comment line. go to the next newline chraracter
CommentLoop:
for {
char, _, err = r.ReadRune()
if err != nil {
return parameter
}
parameter = parameter + string(char)
if char == '\n' {
break CommentLoop
}
}
// Go round again as we're now on a new line
continue NewLineLoop
}
// We've checked for comment and it isn't a comment line so break without moving buffer forwards
break NewLineLoop
}
// Move the buffer forward and try again
char, _, _ = r.ReadRune()
parameter = parameter + string(char)
}
}
}
return parameter
}
// getParameterString reads from r in order to find the end of the MQSC Parameter value. This is enclosed in ( ).
// This function will return what it finds and will increment the reader pointer along as it goes.
func getParameterString(r *bufio.Reader) string {
// Add the ( in as it will have been dropped before.
parameter := "("
Loop:
for {
char, _, err := r.ReadRune()
if err != nil {
return parameter
}
parameter = parameter + string(char)
switch char {
case ')':
break Loop
// TODO: Duplicate code..
case '\'', '"':
parameter = parameter + findEndOfParamterString(char, r)
}
}
return parameter
}
func resetAllParameters(currentVerb, originalString *string, lineContinuation, foundGap, parameterNext, redacting, checkComment *bool) {
*currentVerb = ""
*originalString = ""
*lineContinuation = false
*foundGap = false
*parameterNext = false
*redacting = false
*checkComment = true
}
// Redact is the main function for redacting sensitive parameters in MQSC strings
// It accepts a string and redacts sensitive paramters such as LDAPPWD or PASSWORD
func Redact(out string) (string, error) {
out = strings.TrimSpace(out)
var returnStr, currentVerb, originalString string
var lineContinuation, foundGap, parameterNext, redacting, checkComment bool
newline := true
resetAllParameters(&currentVerb, &originalString, &lineContinuation, &foundGap, &parameterNext, &redacting, &checkComment)
r := bufio.NewReader(strings.NewReader(out))
MainLoop:
for {
// We have found a opening ( so use special parameter parsing
if parameterNext {
parameterStr := getParameterString(r)
if !redacting {
returnStr = returnStr + parameterStr
} else {
returnStr = returnStr + redactionString
}
resetAllParameters(&currentVerb, &originalString, &lineContinuation, &foundGap, &parameterNext, &redacting, &checkComment)
}
// Loop round getting hte next parameter
char, _, err := r.ReadRune()
if err == io.EOF {
if originalString != "" {
returnStr = returnStr + originalString
}
break
} else if err != nil {
return returnStr, err
}
/* We need to push forward until we find a non-whitespace, digit or colon character */
if newline {
startOutput, _ := regexp.MatchString(`[^:0-9\s]`, string(char))
if !startOutput {
originalString = originalString + string(char)
continue MainLoop
}
newline = false
}
switch char {
// Found a line continuation character
case '+', '-':
lineContinuation = true
foundGap = false
originalString = originalString + string(char)
continue MainLoop
// Found whitespace/new line
case '\n':
checkComment = true
newline = true
fallthrough
case '\t', '\r', ' ':
if !lineContinuation {
foundGap = true
}
originalString = originalString + string(char)
continue MainLoop
// Found a paramter value
case '(':
parameterNext = true
/* Do not continue as we need to do some checks */
// Found a comment, parse in a special manner
case '*':
if checkComment {
originalString = originalString + string(char)
// Loop round until we find the new line character that marks the end of the comment
CommentLoop:
for {
char, _, err := r.ReadRune()
if err == io.EOF {
if originalString != "" {
returnStr = returnStr + originalString
}
break MainLoop
} else if err != nil {
return returnStr, err
}
originalString = originalString + string(char)
if char == '\n' {
break CommentLoop
}
}
//Comment has been read and added to original string, go back to start
checkComment = true
newline = true
continue MainLoop
}
/* Do not continue as we need to do some checks */
} //end of switch
checkComment = false
if lineContinuation {
lineContinuation = false
}
if foundGap || parameterNext {
// we've completed an parameter so check whether it is sensitive
currentVerb = strings.ToUpper(currentVerb)
if isSensitiveCommand(currentVerb) {
redacting = true
}
// Add the unedited string to the return string
returnStr = returnStr + originalString
//reset some of the parameters
originalString = ""
currentVerb = ""
foundGap = false
lineContinuation = false
}
originalString = originalString + string(char)
currentVerb = currentVerb + string(char)
}
return returnStr, nil
}
// isSensitiveCommand checks whether the given string contains a sensitive parameter.
// We use contains here because we can't determine whether a line continuation seperates
// parts of a parameter or two different parameters.
func isSensitiveCommand(command string) bool {
for _, v := range sensitiveParameters {
if strings.Contains(command, v) {
return true
}
}
return false
}

View File

@@ -0,0 +1,171 @@
/*
© Copyright IBM Corporation 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package mqscredact
import (
"strings"
"testing"
)
const passwordString = passwordHalf1 + passwordHalf2
const passwordHalf1 = "hippo"
const passwordHalf2 = "123456"
var testStrings = [...]string{
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD('" + passwordString + "')",
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD(\"" + passwordString + "\")",
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD ('" + passwordString + "')",
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD\t\t('" + passwordString + "')",
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) ldappwd('" + passwordString + "')",
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LdApPwD('" + passwordString + "')",
"DEFINE CHANNEL(CHL) CHLTYPE(SOMETHING) PASSWORD('" + passwordString + "')",
"DEFINE CHANNEL(CHL) CHLTYPE(SOMETHING) PASSWORD(\"" + passwordString + "\")",
"DEFINE CHANNEL(CHL) CHLTYPE(SOMETHING) PASSWORD ('" + passwordString + "')",
"DEFINE CHANNEL(CHL) CHLTYPE(SOMETHING) PASSWORD\t\t('" + passwordString + "')",
"DEFINE CHANNEL(CHL) CHLTYPE(SOMETHING) password('" + passwordString + "')",
"DEFINE CHANNEL(CHL) CHLTYPE(SOMETHING) pAsSwOrD('" + passwordString + "')",
"ALTER QMGR SSLCRYP('" + passwordString + "')",
"ALTER QMGR SSLCRYP(\"" + passwordString + "\")",
"ALTER QMGR SSLCRYP ('" + passwordString + "')",
"ALTER QMGR SSLCRYP\t\t('" + passwordString + "')",
"ALTER QMGR sslcryp('" + passwordString + "')",
"ALTER QMGR sslCRYP('" + passwordString + "')",
// Line continuation ones
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD(\"" + passwordHalf1 + "+\n " + passwordHalf2 + "\")",
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD(\"" + passwordHalf1 + "+\n\t" + passwordHalf2 + "\")",
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD(\"" + passwordHalf1 + "+\n\t " + passwordHalf2 + "\")",
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD('" + passwordHalf1 + "+\n " + passwordHalf2 + "')",
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD('" + passwordHalf1 + "+\n\t" + passwordHalf2 + "')",
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD('" + passwordHalf1 + "+\n\t " + passwordHalf2 + "')",
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD(\"" + passwordHalf1 + "-\n" + passwordHalf2 + "\")",
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD('" + passwordHalf1 + "-\n" + passwordHalf2 + "')",
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD(\"" + passwordHalf1 + "+ \n " + passwordHalf2 + "\")",
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD(\"" + passwordHalf1 + "+\t\n " + passwordHalf2 + "\")",
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD(\"" + passwordHalf1 + "- \n" + passwordHalf2 + "\")",
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD(\"" + passwordHalf1 + "-\t\n" + passwordHalf2 + "\")",
"DEFINE CHANNEL(CHL) CHLTYPE(SOMETHING) PASSWORD(\"" + passwordHalf1 + "+\n " + passwordHalf2 + "\")",
"ALTER QMGR SSLCRYP(\"" + passwordHalf1 + "+\n " + passwordHalf2 + "\")",
//edge cases
"ALTER QMGR SSLCRYP(\"" + passwordHalf1 + "+\n 123+\n 456\")",
"ALTER QMGR SSLCRYP(\"" + passwordHalf1 + "-\n123-\n456\")",
"ALTER QMGR SSLCRYP(\"" + passwordHalf1 + "+\n 1+\n 2+\n 3+\n 4+\n 5+\n 6\")",
"ALTER QMGR SSLCRYP(\"" + passwordHalf1 + "-\n1-\n2-\n3-\n4-\n5-\n6\")",
"ALTER QMGR SSLCRYP + \n (\"" + passwordHalf1 + "+\n 1+\n 2+\n 3+\n 4+\n 5+\n 6\")",
"ALTER QMGR SSLCRYP - \n(\"" + passwordHalf1 + "-\n1-\n2-\n3-\n4-\n5-\n6\")",
"ALTER QMGR SSL + \n CRYP(\"" + passwordHalf1 + "+\n 1+\n 2+\n 3+\n 4+\n 5+\n 6\")",
"ALTER QMGR SSL - \nCRYP(\"" + passwordHalf1 + "-\n1-\n2-\n3-\n4-\n5-\n6\")",
"ALTER QMGR + \n SSL +\n CRYP(\"" + passwordHalf1 + "+\n 1+\n 2+\n 3+\n 4+\n 5+\n 6\") +\n TEST(1234)",
"ALTER QMGR -\nSSL -\nCRYP(\"" + passwordHalf1 + "-\n1-\n2-\n3-\n4-\n5-\n6\") -\nTEST(1234)",
"ALTER QMGR +\n * COMMENT\n SSL +\n * COMMENT IN MIDDLE\n CRYP('" + passwordString + "')",
" 1: ALTER CHANNEL(TEST2) CHLTYPE(SDR) PASS+\n : *test comment\n : WORD('" + passwordString + "')",
" 2: ALTER CHANNEL(TEST3) CHLTYPE(SDR) PASSWORD('" + passwordHalf1 + "-\n*commentinmiddle with ' \n" + passwordHalf2 + "')",
" 3: ALTER CHANNEL(TEST3) CHLTYPE(SDR) PASSWORD('" + passwordHalf1 + "-\n*commentinmiddle with ') \n" + passwordHalf2 + "')",
}
var expected = [...]string{
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD" + redactionString,
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD" + redactionString,
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD " + redactionString,
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD\t\t" + redactionString,
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) ldappwd" + redactionString,
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LdApPwD" + redactionString,
"DEFINE CHANNEL(CHL) CHLTYPE(SOMETHING) PASSWORD" + redactionString,
"DEFINE CHANNEL(CHL) CHLTYPE(SOMETHING) PASSWORD" + redactionString,
"DEFINE CHANNEL(CHL) CHLTYPE(SOMETHING) PASSWORD " + redactionString,
"DEFINE CHANNEL(CHL) CHLTYPE(SOMETHING) PASSWORD\t\t" + redactionString,
"DEFINE CHANNEL(CHL) CHLTYPE(SOMETHING) password" + redactionString,
"DEFINE CHANNEL(CHL) CHLTYPE(SOMETHING) pAsSwOrD" + redactionString,
"ALTER QMGR SSLCRYP" + redactionString,
"ALTER QMGR SSLCRYP" + redactionString,
"ALTER QMGR SSLCRYP " + redactionString,
"ALTER QMGR SSLCRYP\t\t" + redactionString,
"ALTER QMGR sslcryp" + redactionString,
"ALTER QMGR sslCRYP" + redactionString,
// Line continuation ones
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD" + redactionString,
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD" + redactionString,
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD" + redactionString,
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD" + redactionString,
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD" + redactionString,
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD" + redactionString,
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD" + redactionString,
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD" + redactionString,
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD" + redactionString,
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD" + redactionString,
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD" + redactionString,
"DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD" + redactionString,
"DEFINE CHANNEL(CHL) CHLTYPE(SOMETHING) PASSWORD" + redactionString,
"ALTER QMGR SSLCRYP" + redactionString,
//edge cases
"ALTER QMGR SSLCRYP" + redactionString,
"ALTER QMGR SSLCRYP" + redactionString,
"ALTER QMGR SSLCRYP" + redactionString,
"ALTER QMGR SSLCRYP" + redactionString,
"ALTER QMGR SSLCRYP + \n \t " + redactionString,
"ALTER QMGR SSLCRYP - \n " + redactionString,
"ALTER QMGR SSL + \n CRYP" + redactionString,
"ALTER QMGR SSL - \nCRYP" + redactionString,
"ALTER QMGR + \n SSL +\n CRYP" + redactionString + " +\n TEST(1234)",
"ALTER QMGR -\nSSL -\nCRYP" + redactionString + " -\nTEST(1234)",
"ALTER QMGR +\n * COMMENT\n SSL +\n * COMMENT IN MIDDLE\n CRYP" + redactionString,
"1: ALTER CHANNEL(TEST2) CHLTYPE(SDR) PASS+\n : *test comment\n : WORD" + redactionString,
"2: ALTER CHANNEL(TEST3) CHLTYPE(SDR) PASSWORD" + redactionString,
"3: ALTER CHANNEL(TEST3) CHLTYPE(SDR) PASSWORD" + redactionString,
}
// Returns true if the 2 strings are equal ignoring whitespace characters
func compareIgnoreWhiteSpace(str1, str2 string) bool {
whiteSpaces := [...]string{" ", "\t", "\n", "\r"}
for _, w := range whiteSpaces {
str1 = strings.Replace(str1, w, "", -1)
str2 = strings.Replace(str2, w, "", -1)
}
return str1 == str2
}
func TestAll(t *testing.T) {
for i, v := range testStrings {
back, _ := Redact(v)
if strings.Contains(back, passwordHalf1) || strings.Contains(back, passwordHalf2) || strings.Contains(back, passwordString) {
t.Errorf("MAJOR FAIL[%d]: Found an instance of the password. ", i)
}
if !compareIgnoreWhiteSpace(back, expected[i]) {
t.Errorf("FAIL[%d]:\nGave :%s\nexpected:%s\ngot :%s", i, v, expected[i], back)
}
}
}

View File

@@ -13,7 +13,9 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
// Package mqtemplate contains code to process template files
package mqtemplate
import (
"os"
@@ -21,11 +23,12 @@ import (
"text/template"
"github.com/ibm-messaging/mq-container/internal/command"
"github.com/ibm-messaging/mq-container/internal/logger"
)
// processTemplateFile takes a Go templateFile, and processes it with the
// ProcessTemplateFile takes a Go templateFile, and processes it with the
// supplied data, writing to destFile
func processTemplateFile(templateFile, destFile string, data interface{}) error {
func ProcessTemplateFile(templateFile, destFile string, data interface{}, log *logger.Logger) error {
// Re-configure channel if app password not set
t, err := template.ParseFiles(templateFile)
if err != nil {

View File

@@ -1,5 +1,5 @@
/*
© Copyright IBM Corporation 2018
© Copyright IBM Corporation 2018, 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -20,6 +20,9 @@ package ready
import (
"io/ioutil"
"os"
"strings"
"github.com/ibm-messaging/mq-container/internal/command"
)
const fileName string = "/run/runmqserver/ready"
@@ -62,3 +65,24 @@ func Check() (bool, error) {
}
return exists, nil
}
// IsRunningAsActiveQM returns true if the queue manager is running in active mode
func IsRunningAsActiveQM(name string) (bool, error) {
return isRunningQM(name, "(RUNNING)")
}
// IsRunningAsStandbyQM returns true if the queue manager is running in standby mode
func IsRunningAsStandbyQM(name string) (bool, error) {
return isRunningQM(name, "(RUNNING AS STANDBY)")
}
func isRunningQM(name string, status string) (bool, error) {
out, _, err := command.Run("dspmq", "-n", "-m", name)
if err != nil {
return false, err
}
if strings.Contains(string(out), status) {
return true, nil
}
return false, nil
}

610
internal/tls/tls.go Normal file
View File

@@ -0,0 +1,610 @@
/*
© Copyright IBM Corporation 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package tls
import (
"bufio"
"fmt"
"io/ioutil"
pwr "math/rand"
"os"
"path/filepath"
"strings"
"time"
"crypto/rand"
"crypto/sha512"
"crypto/x509"
"encoding/pem"
"github.com/ibm-messaging/mq-container/internal/filecheck"
"github.com/ibm-messaging/mq-container/internal/keystore"
pkcs "software.sslmate.com/src/go-pkcs12"
)
// IntegrationDefaultLabel is the default certificate label used by Cloud Integration Platform
const IntegrationDefaultLabel = "default"
// P12TrustStoreName is the name of the PKCS#12 truststore used by the webconsole
const P12TrustStoreName = "trust.p12"
// CMSKeyStoreName is the name of the CMS Keystore used by the queue manager
const CMSKeyStoreName = "key.kdb"
type KeyStoreData struct {
Keystore *keystore.KeyStore
Password string
TrustedCerts []*pem.Block
KnownFingerPrints []string
KeyLabels []string
}
type P12KeyFiles struct {
Keystores []string
Password string
}
func getCertFingerPrint(block *pem.Block) (string, error) {
// Add to future truststore and known certs (if not already there)
cert, err := x509.ParseCertificate(block.Bytes)
if err != nil {
return "", fmt.Errorf("could not parse x509 certificate: %v", err)
}
sha512Sum := sha512.Sum512(cert.Raw)
sha512str := string(sha512Sum[:])
return sha512str, nil
}
// Add to Keystores known certs (if not already there) and add to the
// Keystore if "addToKeystore" is true.
func addCertToKeyData(block *pem.Block, keyData *KeyStoreData, addToKeystore bool) error {
sha512str, err := getCertFingerPrint(block)
if err != nil {
return err
}
known := false
for _, fingerprint := range keyData.KnownFingerPrints {
if fingerprint == sha512str {
known = true
break
}
}
if !known {
// Sometimes we don't want to add to the keystore trust here.
// For example if it will be imported with the key later.
if addToKeystore {
keyData.TrustedCerts = append(keyData.TrustedCerts, block)
}
keyData.KnownFingerPrints = append(keyData.KnownFingerPrints, sha512str)
}
return nil
}
// Generates a random 12 character password from the characters a-z, A-Z, 0-9.
func generateRandomPassword() string {
pwr.Seed(time.Now().Unix())
validChars := "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
validcharArray := []byte(validChars)
password := ""
for i := 0; i < 12; i++ {
password = password + string(validcharArray[pwr.Intn(len(validcharArray))])
}
return password
}
// Creates the PKCS#12 Truststore and the CMS Keystore.
func generateAllStores(dir string) (KeyStoreData, KeyStoreData, error) {
var cmsKeystore, p12TrustStore KeyStoreData
pw := generateRandomPassword()
cmsKeystore.Password = pw
p12TrustStore.Password = pw
// Create the keystore Directory (if it doesn't already exist)
// #nosec G301 - write group permissions are required
err := os.MkdirAll(dir, 0770)
if err != nil {
return cmsKeystore, p12TrustStore, fmt.Errorf("Failed to create keystore directory: %v", err)
}
p12TrustStore.Keystore = keystore.NewPKCS12KeyStore(filepath.Join(dir, P12TrustStoreName), p12TrustStore.Password)
err = p12TrustStore.Keystore.Create()
if err != nil {
return cmsKeystore, p12TrustStore, fmt.Errorf("Failed to create PKCS#12 TrustStore: %v", err)
}
cmsKeystore.Keystore = keystore.NewCMSKeyStore(filepath.Join(dir, CMSKeyStoreName), cmsKeystore.Password)
err = cmsKeystore.Keystore.Create()
if err != nil {
return cmsKeystore, p12TrustStore, fmt.Errorf("Failed to create CMS KeyStore: %v", err)
}
return cmsKeystore, p12TrustStore, nil
}
// processKeys walks through the keyDir directory and imports any keys it finds to individual PKCS#12 keystores
// and the CMS KeyStore. The label it uses is the name of the directory if finds the keys in.
func processKeys(keyDir, outputDir string, cmsKeyDB, p12TrustDB *KeyStoreData) (string, P12KeyFiles, error) {
var p12s P12KeyFiles
var firstLabel string
pwToUse := cmsKeyDB.Password
p12s.Password = pwToUse
trustStoreReserveredName := P12TrustStoreName[0 : len(P12TrustStoreName)-len(filepath.Ext(P12TrustStoreName))]
keyList, err := ioutil.ReadDir(keyDir)
if err == nil && len(keyList) > 0 {
// Found some keys, verify the contents
for _, key := range keyList {
keys, _ := ioutil.ReadDir(filepath.Join(keyDir, key.Name()))
keyLabel := key.Name()
if keyLabel == trustStoreReserveredName {
return firstLabel, p12s, fmt.Errorf("Found key with same label set as same name as truststore(%s). This is not allowed", trustStoreReserveredName)
}
keyfilename := ""
var keyfile interface{}
var certFile *x509.Certificate
var caFile []*x509.Certificate
// find the keyfile name
for _, a := range keys {
if strings.HasSuffix(a.Name(), ".key") {
// #nosec G304 - filename variable is derived from contents of 'keyDir' which is a defined constant
keyFile, err := ioutil.ReadFile(filepath.Join(keyDir, key.Name(), a.Name()))
if err != nil {
return firstLabel, p12s, fmt.Errorf("Could not read keyfile %s: %v", filepath.Join(keyDir, key.Name(), a.Name()), err)
}
block, _ := pem.Decode(keyFile)
if block == nil {
return firstLabel, p12s, fmt.Errorf("Could not decode keyfile %s: pem.Decode returned nil", filepath.Join(keyDir, key.Name(), a.Name()))
}
//Test whether it is PKCS1
keyfile, err = x509.ParsePKCS1PrivateKey(block.Bytes)
if err != nil {
// Before we fail check whether it is PKCS8
keyfile, err = x509.ParsePKCS8PrivateKey(block.Bytes)
if err != nil {
fmt.Printf("key %s ParsePKCS1/8PrivateKey ERR: %v\n", filepath.Join(keyDir, key.Name(), a.Name()), err)
return firstLabel, p12s, err
}
//It was PKCS8 afterall
}
keyfilename = a.Name()
}
}
if keyfile == nil {
continue
}
// Find out what the keyfile was called without the extension
prefix := keyfilename[0 : len(keyfilename)-len(filepath.Ext(keyfilename))]
for _, a := range keys {
if strings.HasSuffix(a.Name(), ".key") {
continue
}
if strings.HasPrefix(a.Name(), prefix) && strings.HasSuffix(a.Name(), ".crt") {
// #nosec G304 - filename variable is derived from contents of 'keyDir' which is a defined constant
cert, err := ioutil.ReadFile(filepath.Join(keyDir, key.Name(), a.Name()))
if err != nil {
return firstLabel, p12s, fmt.Errorf("Could not read file %s: %v", filepath.Join(keyDir, key.Name(), a.Name()), err)
}
block, _ := pem.Decode(cert)
if block == nil {
return firstLabel, p12s, fmt.Errorf("Could not decode certificate %s: pem.Decode returned nil", filepath.Join(keyDir, key.Name(), a.Name()))
}
certFile, err = x509.ParseCertificate(block.Bytes)
if err != nil {
return firstLabel, p12s, fmt.Errorf("Could not parse certificate %s: %v", filepath.Join(keyDir, key.Name(), a.Name()), err)
}
// Add to the dup list for the CMS keystore but not the PKCS#12 Truststore
err = addCertToKeyData(block, cmsKeyDB, false)
} else if strings.HasSuffix(a.Name(), ".crt") {
// #nosec G304 - filename variable is derived from contents of 'keyDir' which is a defined constant
remainder, err := ioutil.ReadFile(filepath.Join(keyDir, key.Name(), a.Name()))
if err != nil {
return firstLabel, p12s, fmt.Errorf("Could not read file %s: %v", filepath.Join(keyDir, key.Name(), a.Name()), err)
}
for string(remainder) != "" {
var block *pem.Block
block, remainder = pem.Decode(remainder)
// If we can't decode the CA certificate then just exit.
if block == nil {
break
}
// Add to the dup list for the CMS keystore
err = addCertToKeyData(block, cmsKeyDB, false)
// Add to the p12 truststore
err = addCertToKeyData(block, p12TrustDB, true)
caCert, err := x509.ParseCertificate(block.Bytes)
if err != nil {
return firstLabel, p12s, fmt.Errorf("Could not parse CA certificate %s: %v", filepath.Join(keyDir, key.Name(), a.Name()), err)
}
caFile = append(caFile, caCert)
}
}
}
// Create p12 keystore
file, err := pkcs.Encode(rand.Reader, keyfile, certFile, caFile, pwToUse)
if err != nil {
return firstLabel, p12s, fmt.Errorf("Could not encode PKCS#12 Keystore %s: %v", keyLabel+".p12", err)
}
err = ioutil.WriteFile(filepath.Join(outputDir, keyLabel+".p12"), file, 0644)
if err != nil {
return firstLabel, p12s, fmt.Errorf("Could not write PKCS#12 Keystore %s: %v", filepath.Join(outputDir, keyLabel+".p12"), err)
}
p12s.Keystores = append(p12s.Keystores, keyLabel+".p12")
// Add to the CMS keystore
err = cmsKeyDB.Keystore.Import(filepath.Join(outputDir, keyLabel+".p12"), pwToUse)
if err != nil {
return firstLabel, p12s, fmt.Errorf("Could not import keys from %s into CMS Keystore: %v", filepath.Join(outputDir, keyLabel+".p12"), err)
}
// Relabel it
allLabels, err := cmsKeyDB.Keystore.GetCertificateLabels()
if err != nil {
return firstLabel, p12s, fmt.Errorf("Could not list keys in CMS Keystore: %v", err)
}
relabelled := false
for _, cl := range allLabels {
found := false
for _, kl := range cmsKeyDB.KeyLabels {
if strings.Trim(cl, "\"") == kl {
found = true
break
}
}
if !found {
// This is the one to rename
err = cmsKeyDB.Keystore.RenameCertificate(strings.Trim(cl, "\""), keyLabel)
if err != nil {
return firstLabel, p12s, err
}
relabelled = true
cmsKeyDB.KeyLabels = append(cmsKeyDB.KeyLabels, keyLabel)
break
}
}
if !relabelled {
return firstLabel, p12s, fmt.Errorf("Unable to find the added key for %s in CMS keystore", keyLabel)
}
// First key found so mark it as the one to use with the queue manager.
if firstLabel == "" {
firstLabel = keyLabel
}
}
}
return firstLabel, p12s, nil
}
// processTrustCertificates walks through the trustDir directory and adds any certificates it finds
// to the PKCS#12 Truststore and the CMS KeyStore as long as has not already been added.
func processTrustCertificates(trustDir string, cmsKeyDB, p12TrustDB *KeyStoreData) error {
certList, err := ioutil.ReadDir(trustDir)
if err == nil && len(certList) > 0 {
// Found some keys, verify the contents
for _, cert := range certList {
certs, _ := ioutil.ReadDir(filepath.Join(trustDir, cert.Name()))
for _, a := range certs {
if strings.HasSuffix(a.Name(), ".crt") {
// #nosec G304 - filename variable is derived from contents of 'trustDir' which is a defined constant
remainder, err := ioutil.ReadFile(filepath.Join(trustDir, cert.Name(), a.Name()))
if err != nil {
return fmt.Errorf("Could not read file %s: %v", filepath.Join(trustDir, cert.Name(), a.Name()), err)
}
for string(remainder) != "" {
var block *pem.Block
block, remainder = pem.Decode(remainder)
if block == nil {
break
}
// Add to the CMS keystore
err = addCertToKeyData(block, cmsKeyDB, true)
// Add to the p12 truststore
err = addCertToKeyData(block, p12TrustDB, true)
}
}
}
}
}
// We've potentially created two lists of certificates to import. Add them both to relevant Truststores
if len(p12TrustDB.TrustedCerts) > 0 {
// Do P12 TrustStore first
temporaryPemFile := filepath.Join("/tmp", "trust.pem")
_, err := os.Stat(temporaryPemFile)
if err == nil {
err = os.Remove(temporaryPemFile)
if err != nil {
return fmt.Errorf("Could not remove file %v: %v", temporaryPemFile, err)
}
}
err = writeCertsToFile(temporaryPemFile, p12TrustDB.TrustedCerts)
if err != nil {
return err
}
err = p12TrustDB.Keystore.AddNoLabel(temporaryPemFile)
if err != nil {
return fmt.Errorf("Could not add certificates to PKCS#12 Truststore: %v", err)
}
// We need to relabel everything because liberty doesn't play nicely with autolabelled certs
allCerts, err := p12TrustDB.Keystore.ListAllCertificates()
if err != nil || len(allCerts) <= 0 {
return fmt.Errorf("Could not get any certificates from PKCS#12 Truststore: %v", err)
}
for i, cert := range allCerts {
cert = strings.Trim(cert, "\"")
cert = strings.TrimSpace(cert)
newLabel := fmt.Sprintf("Trust%d", i)
err = p12TrustDB.Keystore.RenameCertificate(cert, newLabel)
if err != nil || len(allCerts) <= 0 {
return fmt.Errorf("Could not rename certificate %s to %s in PKCS#12 Truststore: %v", cert, newLabel, err)
}
}
}
if len(cmsKeyDB.TrustedCerts) > 0 {
// Now the CMS Keystore
temporaryPemFile := filepath.Join("/tmp", "cmsTrust.pem")
_, err := os.Stat(temporaryPemFile)
if err == nil {
err = os.Remove(temporaryPemFile)
if err != nil {
return fmt.Errorf("Could not remove file %v: %v", temporaryPemFile, err)
}
}
err = writeCertsToFile(temporaryPemFile, cmsKeyDB.TrustedCerts)
if err != nil {
return err
}
err = cmsKeyDB.Keystore.AddNoLabel(temporaryPemFile)
if err != nil {
return fmt.Errorf("Could not add certificates to CMS keystore: %v", err)
}
}
return nil
}
// Writes a given list of certificates to a file.
func writeCertsToFile(file string, certs []*pem.Block) error {
f, err := os.Create(file)
if err != nil {
return fmt.Errorf("writeCertsToFile: Could not create file %s: %v", file, err)
}
defer f.Close()
w := bufio.NewWriter(f)
for i, c := range certs {
err := pem.Encode(w, c)
if err != nil {
return fmt.Errorf("writeCertsToFile: Could not encode certificate number %d: %v", i, err)
}
err = w.Flush()
if err != nil {
return fmt.Errorf("writeCertsToFile: Failed to write to file %s: %v", file, err)
}
}
return nil
}
// ConfigureTLSKeystores sets up the TLS Trust and Keystores for use
func ConfigureTLSKeystores(keyDir, certDir, outputDir string) (string, KeyStoreData, KeyStoreData, P12KeyFiles, error) {
var returnLabel, label string
var cmsKeyDB, p12TrustDB KeyStoreData
var keyFiles P12KeyFiles
var err error
cmsKeyDB, p12TrustDB, err = generateAllStores(outputDir)
if err != nil {
return returnLabel, cmsKeyDB, p12TrustDB, keyFiles, err
}
returnLabel, err = expandOldTLSVariable(keyDir, outputDir, &cmsKeyDB, &p12TrustDB)
if err != nil {
return returnLabel, cmsKeyDB, p12TrustDB, keyFiles, err
}
label, keyFiles, err = processKeys(keyDir, outputDir, &cmsKeyDB, &p12TrustDB)
if err != nil {
return returnLabel, cmsKeyDB, p12TrustDB, keyFiles, err
}
if returnLabel == "" {
returnLabel = label
}
err = processTrustCertificates(certDir, &cmsKeyDB, &p12TrustDB)
if err != nil {
return returnLabel, cmsKeyDB, p12TrustDB, keyFiles, err
}
return returnLabel, cmsKeyDB, p12TrustDB, keyFiles, err
}
// This function supports the old mechanism of importing certificates supplied by the MQ_TLS_KEYSTORE envvar
func expandOldTLSVariable(keyDir, outputDir string, cmsKeyDB, p12TrustDB *KeyStoreData) (string, error) {
// TODO: Change this or find a way to set it
outputDirName := "acopiedcertificate"
// Check whether the old variable is set. If not exit quietly
keyfile := os.Getenv("MQ_TLS_KEYSTORE")
if keyfile == "" {
return "", nil
}
// There is a file to read and process
keyfilepw := os.Getenv("MQ_TLS_PASSPHRASE")
if !strings.HasSuffix(keyfile, ".p12") {
return "", fmt.Errorf("MQ_TLS_KEYSTORE (%s) does not point to a PKCS#12 file ending with the suffix .p12", keyfile)
}
_, err := os.Stat(keyfile)
if err != nil {
return "", fmt.Errorf("File %s referenced by MQ_TLS_KEYSTORE does not exist", keyfile)
}
err = filecheck.CheckFileSource(keyfile)
if err != nil {
return "", fmt.Errorf("File %s referenced by MQ_TLS_KEYSTORE is invalid: %v", keyfile, err)
}
// #nosec G304 - filename variable 'keyfile' is checked above to ensure it is valid
readkey, err := ioutil.ReadFile(keyfile)
if err != nil {
return "", fmt.Errorf("Failed to read %s: %v", keyfile, err)
}
// File has been checked and read, decode it.
pk, cert, cas, err := pkcs.DecodeChain(readkey, keyfilepw)
if err != nil {
return "", fmt.Errorf("Failed to decode %s: %v", keyfile, err)
}
// Find a directory name that doesn't exist
for {
_, err := os.Stat(filepath.Join(keyDir, outputDirName))
if err == nil {
outputDirName = outputDirName + "0"
} else {
break
}
}
//Bceause they supplied this certificate using the old method we should use this for qm & webconsole
overrideLabel := outputDirName
// Write out the certificate for the private key
if cert != nil {
block := pem.Block{
Type: "CERTIFICATE",
Headers: nil,
Bytes: cert.Raw,
}
err = addCertToKeyData(&block, cmsKeyDB, false)
if err != nil {
return "", fmt.Errorf("expandOldTLSVariable: Failed to add cert to CMS Keystore duplicate list: %v", err)
}
err = addCertToKeyData(&block, p12TrustDB, true)
if err != nil {
return "", fmt.Errorf("expandOldTLSVariable: Failed to add cert to P12 Truststore duplicate list: %v", err)
}
}
// now write out all the ca certificates
if cas != nil || len(cas) > 0 {
for i, c := range cas {
block := pem.Block{
Type: "CERTIFICATE",
Headers: nil,
Bytes: c.Raw,
}
// Add to the dup list for the CMS keystore
err = addCertToKeyData(&block, cmsKeyDB, false)
if err != nil {
return "", fmt.Errorf("expandOldTLSVariable: Failed to add CA cert %d to CMS Keystore duplicate list: %v", i, err)
}
// Add to the p12 truststore
err = addCertToKeyData(&block, p12TrustDB, true)
if err != nil {
return "", fmt.Errorf("expandOldTLSVariable: Failed to add CA cert %d to P12 Truststore duplicate list: %v", i, err)
}
}
}
// Now we've handled the certificates copy the keystore into place
destination := filepath.Join(outputDir, outputDirName+".p12")
// Create p12 keystore
file, err := pkcs.Encode(rand.Reader, pk, cert, cas, p12TrustDB.Password)
if err != nil {
return "", fmt.Errorf("Failed to re-encode p12 keystore: %v", err)
}
err = ioutil.WriteFile(destination, file, 0644)
if err != nil {
return "", fmt.Errorf("Failed to write p12 keystore: %v", err)
}
// Add to the CMS keystore
err = cmsKeyDB.Keystore.Import(destination, p12TrustDB.Password)
if err != nil {
return "", fmt.Errorf("Failed to import p12 keystore %s: %v", destination, err)
}
if pk != nil {
// Relabel the key
allLabels, err := cmsKeyDB.Keystore.GetCertificateLabels()
if err != nil {
fmt.Printf("cms GetCertificateLabels: %v\n", err)
return "", err
}
relabelled := false
for _, cl := range allLabels {
found := false
for _, kl := range cmsKeyDB.KeyLabels {
if strings.Trim(cl, "\"") == kl {
found = true
break
}
}
if !found {
// This is the one to rename
err = cmsKeyDB.Keystore.RenameCertificate(strings.Trim(cl, "\""), outputDirName)
if err != nil {
return "", err
}
relabelled = true
cmsKeyDB.KeyLabels = append(cmsKeyDB.KeyLabels, outputDirName)
break
}
}
if !relabelled {
return "", fmt.Errorf("Unable to find the added key in CMS keystore")
}
}
return overrideLabel, nil
}

View File

@@ -0,0 +1,28 @@
# © Copyright IBM Corporation 2019
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
image: ibmcom/mq:9.1.2.0-UBI
manifests:
- image: ibmcom/mq:9.1.2.0-UBI-amd64
platform:
architecture: amd64
os: linux
- image: ibmcom/mq:9.1.2.0-UBI-ppc64le
platform:
architecture: ppc64le
os: linux
- image: ibmcom/mq:9.1.2.0-UBI-s390x
platform:
architecture: s390x
os: linux

View File

@@ -0,0 +1,28 @@
# © Copyright IBM Corporation 2019
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
image: ibmcom/mq:9.1.3.0-r2
manifests:
- image: ibmcom/mq:9.1.3.0-r2-amd64
platform:
architecture: amd64
os: linux
- image: ibmcom/mq:9.1.3.0-r2-ppc64le
platform:
architecture: ppc64le
os: linux
- image: ibmcom/mq:9.1.3.0-r2-s390x
platform:
architecture: s390x
os: linux

View File

@@ -14,16 +14,15 @@
image: ibmcom/mq:latest
manifests:
- image: ibmcom/mq:9.1.2.0-x86_64
- image: ibmcom/mq:9.1.3.0-r2-amd64
platform:
architecture: amd64
os: linux
- image: ibmcom/mq:9.1.2.0-ppc64le
- image: ibmcom/mq:9.1.3.0-r2-ppc64le
platform:
architecture: ppc64le
os: linux
- image: ibmcom/mq:9.1.2.0-s390x
- image: ibmcom/mq:9.1.3.0-r2-s390x
platform:
architecture: s390x
os: linux

View File

@@ -0,0 +1,28 @@
# © Copyright IBM Corporation 2019
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
image: ibmcorp/mqadvanced-server-dev:9.1.2.0-UBI
manifests:
- image: ibmcorp/mqadvanced-server-dev:9.1.2.0-UBI-amd64
platform:
architecture: amd64
os: linux
- image: ibmcorp/mqadvanced-server-dev:9.1.2.0-UBI-ppc64le
platform:
architecture: ppc64le
os: linux
- image: ibmcorp/mqadvanced-server-dev:9.1.2.0-UBI-s390x
platform:
architecture: s390x
os: linux

View File

@@ -0,0 +1,28 @@
# © Copyright IBM Corporation 2019
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
image: ibmcorp/mqadvanced-server-dev:9.1.3.0-r2
manifests:
- image: ibmcorp/mqadvanced-server-dev:9.1.3.0-r2-amd64
platform:
architecture: amd64
os: linux
- image: ibmcorp/mqadvanced-server-dev:9.1.3.0-r2-ppc64le
platform:
architecture: ppc64le
os: linux
- image: ibmcorp/mqadvanced-server-dev:9.1.3.0-r2-s390x
platform:
architecture: s390x
os: linux

View File

@@ -1,3 +1,5 @@
# RHEL-based container build
Build scripts for building a container image based on Red Hat Enterprise Linux (RHEL), using the [`buildah`](https://github.com/containers/buildah) tool. buildah is supported on RHEL V7.5 and greater.
Build scripts for building a container image based on Red Hat Enterprise Linux (RHEL), using the [`buildah`](https://github.com/containers/buildah) tool. buildah is supported on RHEL V7.5 and greater.
WARNING: The code in this directory is not currently in use, pending deletion. The MQ container is now built using a Red Hat UBI image, using "podman build".

View File

@@ -121,6 +121,13 @@ buildah run --user root $ctr_mq -- chmod 0660 /run/termination-log
install --mode 0550 --owner root --group root ./mq-advanced-server-rhel/writePackages.sh ${mnt_mq}/usr/local/bin/writePackages
buildah run --user root $ctr_mq -- /usr/local/bin/writePackages
# Copy web XML files
cp -R web ${mnt_mq}/etc/mqm/web
# Make "mqm" the owner of all the config files
chown --recursive ${mqm_uid}:${mqm_gid} ${mnt_mq}/etc/mqm/*
chmod --recursive 0750 ${mnt_mq}/etc/mqm/*
###############################################################################
# Final Buildah commands
###############################################################################
@@ -138,8 +145,9 @@ fi
buildah config \
--port 1414/tcp \
--port 9157/tcp \
--port 9443/tcp \
--os linux \
--label architecture=x86_64 \
--label architecture=amd64 \
--label io.openshift.tags="$OSTAG" \
--label io.k8s.display-name="$DISNAME" \
--label io.k8s.description="IBM MQ is messaging middleware that simplifies and accelerates the integration of diverse applications and business data across multiple platforms. It uses message queues to facilitate the exchanges of information and offers a single messaging solution for cloud, mobile, Internet of Things (IoT) and on-premises environments." \

View File

@@ -78,8 +78,7 @@ install --directory --mode 0775 --owner ${mqm_uid} --group 0 ${mnt_mq}/run/runmq
cp ./incubating/mqadvanced-server-dev/*.tpl ${mnt_mq}/etc/mqm/
# Copy web XML files for default developer configuration
mkdir --parents ${mnt_mq}/etc/mqm/web
cp --recursive ./incubating/mqadvanced-server-dev/web/* ${mnt_mq}/etc/mqm/web/
cp -R incubating/mqadvanced-server-dev/web/ ${mnt_mq}/etc/mqm/web
# Make "mqm" the owner of all the config files
chown --recursive ${mqm_uid}:${mqm_gid} ${mnt_mq}/etc/mqm/*
@@ -94,7 +93,7 @@ buildah config \
--port 9157/tcp \
--port 9443/tcp \
--os linux \
--label architecture=x86_64 \
--label architecture=amd64 \
--label io.openshift.tags="mq messaging developer" \
--label io.k8s.display-name="IBM MQ Advanced Server Developer Edition" \
--label io.k8s.description="IBM MQ is messaging middleware that simplifies and accelerates the integration of diverse applications and business data across multiple platforms. It uses message queues to facilitate the exchanges of information and offers a single messaging solution for cloud, mobile, Internet of Things (IoT) and on-premises environments." \

View File

@@ -1,7 +1,7 @@
// +build mqdev
/*
© Copyright IBM Corporation 2018
© Copyright IBM Corporation 2018, 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@@ -138,7 +138,7 @@ func TestDevWebDisabled(t *testing.T) {
Env: []string{
"LICENSE=accept",
"MQ_QMGR_NAME=qm1",
"MQ_DISABLE_WEB_CONSOLE=true",
"MQ_ENABLE_EMBEDDED_WEB_SERVER=false",
},
}
id := runContainer(t, cli, &containerConfig)

View File

@@ -24,7 +24,6 @@ import (
"os"
"path/filepath"
"regexp"
"runtime"
"strconv"
"strings"
"testing"
@@ -34,8 +33,6 @@ import (
"github.com/docker/docker/api/types/network"
"github.com/docker/docker/client"
"github.com/docker/go-connections/nat"
"github.com/ibm-messaging/mq-container/internal/command"
)
func TestLicenseNotSet(t *testing.T) {
@@ -112,75 +109,41 @@ func goldenPath(t *testing.T, metric bool) {
stopContainer(t, cli, id)
}
// TestSecurityVulnerabilitiesUbuntu checks for any vulnerabilities in the image, as reported
// by Ubuntu
func TestSecurityVulnerabilitiesUbuntu(t *testing.T) {
t.Parallel()
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
rc, _ := runContainerOneShot(t, cli, "bash", "-c", "test -d /etc/apt")
if rc != 0 {
t.Skip("Skipping test because container is not Ubuntu-based")
}
// Override the entrypoint to make "apt" only receive security updates, then check for updates
var url string
if runtime.GOARCH == "amd64" {
url = "http://security.ubuntu.com/ubuntu/"
} else {
url = "http://ports.ubuntu.com/ubuntu-ports/"
}
rc, log := runContainerOneShot(t, cli, "bash", "-c", "source /etc/os-release && echo \"deb "+url+" ${VERSION_CODENAME}-security main restricted\" > /etc/apt/sources.list && apt-get update 2>&1 >/dev/null && apt-get --simulate -qq upgrade")
if rc != 0 {
t.Fatalf("Expected success, got %v", rc)
}
lines := strings.Split(strings.TrimSpace(log), "\n")
if len(lines) > 0 && lines[0] != "" {
t.Errorf("Expected no vulnerabilities, found the following:\n%v", log)
}
}
// TestSecurityVulnerabilitiesRedHat checks for any vulnerabilities in the image, as reported
// TestSecurityVulnerabilities checks for any vulnerabilities in the image, as reported
// by Red Hat
func TestSecurityVulnerabilitiesRedHat(t *testing.T) {
func TestSecurityVulnerabilities(t *testing.T) {
t.Parallel()
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
_, ret, _ := command.Run("bash", "-c", "test -f /etc/redhat-release")
if ret != 0 {
t.Skip("Skipping test because host is not RedHat-based")
}
rc, _ := runContainerOneShot(t, cli, "bash", "-c", "test -f /etc/redhat-release")
rc, _ := runContainerOneShot(t, cli, "bash", "-c", "command -v microdnf && test -e /etc/yum.repos.d/ubi.repo")
if rc != 0 {
t.Skip("Skipping test because container is not RedHat-based")
}
id, _, err := command.Run("sudo", "buildah", "from", imageName())
if err != nil {
t.Log(id)
t.Fatal(err)
}
id = strings.TrimSpace(id)
defer command.Run("buildah", "rm", id)
mnt, _, err := command.Run("sudo", "buildah", "mount", id)
if err != nil {
t.Log(mnt)
t.Fatal(err)
}
mnt = strings.TrimSpace(mnt)
out, _, err := command.Run("bash", "-c", "sudo cp /etc/yum.repos.d/* "+filepath.Join(mnt, "/etc/yum.repos.d/"))
if err != nil {
t.Log(out)
t.Fatal(err)
}
out, ret, _ = command.Run("bash", "-c", "yum --installroot="+mnt+" updateinfo list sec | grep /Sec")
if ret != 1 {
t.Errorf("Expected no vulnerabilities, found the following:\n%v", out)
t.Skip("Skipping test because container is based on ubi-minimal, which doesn't include yum")
}
// id, _, err := command.Run("sudo", "buildah", "from", imageName())
// if err != nil {
// t.Log(id)
// t.Fatal(err)
// }
// id = strings.TrimSpace(id)
// defer command.Run("buildah", "rm", id)
// mnt, _, err := command.Run("sudo", "buildah", "mount", id)
// if err != nil {
// t.Log(mnt)
// t.Fatal(err)
// }
// mnt = strings.TrimSpace(mnt)
// out, _, err := command.Run("bash", "-c", "sudo cp /etc/yum.repos.d/* "+filepath.Join(mnt, "/etc/yum.repos.d/"))
// if err != nil {
// t.Log(out)
// t.Fatal(err)
// }
// out, ret, _ := command.Run("bash", "-c", "yum --installroot="+mnt+" updateinfo list sec | grep /Sec")
// if ret != 1 {
// t.Errorf("Expected no vulnerabilities, found the following:\n%v", out)
// }
}
func utilTestNoQueueManagerName(t *testing.T, hostName string, expectedName string) {
@@ -235,7 +198,7 @@ func withVolume(t *testing.T, metric bool) {
if err != nil {
t.Fatal(err)
}
vol := createVolume(t, cli)
vol := createVolume(t, cli, t.Name())
defer removeVolume(t, cli, vol.Name)
containerConfig := container.Config{
Image: imageName(),
@@ -273,6 +236,62 @@ func withVolume(t *testing.T, metric bool) {
waitForReady(t, cli, ctr2.ID)
}
// TestWithSplitVolumesLogsData starts a queue manager with separate log/data mounts
func TestWithSplitVolumesLogsData(t *testing.T) {
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
qmsharedlogs := createVolume(t, cli, "qmsharedlogs")
defer removeVolume(t, cli, qmsharedlogs.Name)
qmshareddata := createVolume(t, cli, "qmshareddata")
defer removeVolume(t, cli, qmshareddata.Name)
err, qmID, qmVol := startMultiVolumeQueueManager(t, cli, true, qmsharedlogs.Name, qmshareddata.Name, []string{"LICENSE=accept", "MQ_QMGR_NAME=qm1"})
defer removeVolume(t, cli, qmVol)
defer cleanContainer(t, cli, qmID)
waitForReady(t, cli, qmID)
}
// TestWithSplitVolumesLogsOnly starts a queue manager with a separate log mount
func TestWithSplitVolumesLogsOnly(t *testing.T) {
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
qmsharedlogs := createVolume(t, cli, "qmsharedlogs")
defer removeVolume(t, cli, qmsharedlogs.Name)
err, qmID, qmVol := startMultiVolumeQueueManager(t, cli, true, qmsharedlogs.Name, "", []string{"LICENSE=accept", "MQ_QMGR_NAME=qm1"})
defer removeVolume(t, cli, qmVol)
defer cleanContainer(t, cli, qmID)
waitForReady(t, cli, qmID)
}
// TestWithSplitVolumesDataOnly starts a queue manager with a separate data mount
func TestWithSplitVolumesDataOnly(t *testing.T) {
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
qmshareddata := createVolume(t, cli, "qmshareddata")
defer removeVolume(t, cli, qmshareddata.Name)
err, qmID, qmVol := startMultiVolumeQueueManager(t, cli, true, "", qmshareddata.Name, []string{"LICENSE=accept", "MQ_QMGR_NAME=qm1"})
defer removeVolume(t, cli, qmVol)
defer cleanContainer(t, cli, qmID)
waitForReady(t, cli, qmID)
}
// TestNoVolumeWithRestart ensures a queue manager container can be stopped
// and restarted cleanly
func TestNoVolumeWithRestart(t *testing.T) {
@@ -298,12 +317,11 @@ func TestNoVolumeWithRestart(t *testing.T) {
// where `runmqserver -i` is run to initialize the storage. Then the
// container can be run as normal.
func TestVolumeRequiresRoot(t *testing.T) {
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
vol := createVolume(t, cli)
vol := createVolume(t, cli, t.Name())
defer removeVolume(t, cli, vol.Name)
// Set permissions on the volume to only allow root to write it
@@ -439,7 +457,7 @@ func TestVolumeUnmount(t *testing.T) {
if err != nil {
t.Fatal(err)
}
vol := createVolume(t, cli)
vol := createVolume(t, cli, t.Name())
defer removeVolume(t, cli, vol.Name)
containerConfig := container.Config{
Image: imageName(),
@@ -554,41 +572,237 @@ func TestMQSC(t *testing.T) {
}
}
// TestLargeMQSC creates a new image with a large MQSC file in, starts a container based
// on that image, and checks that the MQSC has been applied correctly.
func TestLargeMQSC(t *testing.T) {
t.Parallel()
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
const numQueues = 1000
var buf bytes.Buffer
for i := 1; i <= numQueues; i++ {
fmt.Fprintf(&buf, "* Test processing of a large MQSC file, defining queue test%v\nDEFINE QLOCAL(test%v)\n", i, i)
}
var files = []struct {
Name, Body string
}{
{"Dockerfile", fmt.Sprintf(`
FROM %v
USER root
RUN rm -f /etc/mqm/*.mqsc
ADD test.mqsc /etc/mqm/
RUN chmod 0660 /etc/mqm/test.mqsc
USER mqm`, imageName())},
{"test.mqsc", buf.String()},
}
tag := createImage(t, cli, files)
defer deleteImage(t, cli, tag)
containerConfig := container.Config{
Env: []string{"LICENSE=accept", "MQ_QMGR_NAME=qm1"},
Image: tag,
}
id := runContainer(t, cli, &containerConfig)
defer cleanContainer(t, cli, id)
waitForReady(t, cli, id)
rc, mqscOutput := execContainer(t, cli, id, "mqm", []string{"bash", "-c", "echo 'DISPLAY QLOCAL(test" + strconv.Itoa(numQueues) + ")' | runmqsc"})
if rc != 0 {
r := regexp.MustCompile("AMQ[0-9][0-9][0-9][0-9]E")
t.Fatalf("Expected runmqsc to exit with rc=0, got %v with error %v", rc, r.FindString(mqscOutput))
}
}
// TestRedactValidMQSC creates a new image with a Valid MQSC file that contains sensitive information, starts a container based
// on that image, and checks that the MQSC has been redacted in the logs.
func TestRedactValidMQSC(t *testing.T) {
t.Parallel()
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
var buf bytes.Buffer
passwords := "hippoman4567"
sslcryp := fmt.Sprintf("GSK_PKCS11=/usr/lib/pkcs11/PKCS11_API.so;token-label;%s;SYMMETRIC_CIPHER_ON;", passwords)
/* LDAPPWD*/
fmt.Fprintf(&buf, "DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) CONNAME('test(24)') SHORTUSR('sn') LDAPUSER('user') LDAPPWD('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) ldappwd('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) lDaPpWd('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD \t('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) +\n LDAP+\n PWD('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) -\nLDAPP-\nWD('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) +\n*test comment\n LDAPP-\n*test comment2\nWD('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD(%v)\n", passwords)
/* PASSWORD */
fmt.Fprintf(&buf, "DEFINE CHANNEL(TEST2) CHLTYPE(SDR) CONNAME('test(24)') XMITQ('fake') PASSWORD('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER CHANNEL(TEST2) CHLTYPE(SDR) password('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER CHANNEL(TEST2) CHLTYPE(SDR) pAsSwOrD('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER CHANNEL(TEST2) CHLTYPE(SDR) PASSWORD \t('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER CHANNEL(TEST2) +\n CHLTYPE(SDR) PASS+\n WORD+\n ('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER CHANNEL(TEST2) -\nCHLTYPE(SDR) PASS-\nWORD-\n('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER CHANNEL(TEST2) +\n CHLTYPE(SDR) PASS-\n*comemnt 2\nWORD+\n*test comment\n ('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER CHANNEL(TEST2) CHLTYPE(SDR) PASSWORD(%s)\n", passwords)
/* SSLCRYP */
fmt.Fprintf(&buf, "ALTER QMGR SSLCRYP('%v')\n", sslcryp)
fmt.Fprintf(&buf, "ALTER QMGR sslcryp('%v')\n", sslcryp)
fmt.Fprintf(&buf, "ALTER QMGR SsLcRyP('%v')\n", sslcryp)
fmt.Fprintf(&buf, "ALTER QMGR SSLCRYP \t('%v')\n", sslcryp)
fmt.Fprintf(&buf, "ALTER QMGR +\n SSL+\n CRYP+\n ('%v')\n", sslcryp)
fmt.Fprintf(&buf, "ALTER QMGR -\nSSLC-\nRYP-\n('%v')\n", sslcryp)
fmt.Fprintf(&buf, "ALTER QMGR +\n*commenttime\n SSL-\n*commentagain\nCRYP+\n*last comment\n ('%v')\n", sslcryp)
var files = []struct {
Name, Body string
}{
{"Dockerfile", fmt.Sprintf(`
FROM %v
USER root
RUN rm -f /etc/mqm/*.mqsc
ADD test.mqsc /etc/mqm/
RUN chmod 0660 /etc/mqm/test.mqsc
USER mqm`, imageName())},
{"test.mqsc", buf.String()},
}
tag := createImage(t, cli, files)
defer deleteImage(t, cli, tag)
containerConfig := container.Config{
Env: []string{"LICENSE=accept", "MQ_QMGR_NAME=qm1"},
Image: tag,
}
id := runContainer(t, cli, &containerConfig)
defer cleanContainer(t, cli, id)
waitForReady(t, cli, id)
stopContainer(t, cli, id)
scanner := bufio.NewScanner(strings.NewReader(inspectLogs(t, cli, id)))
for scanner.Scan() {
s := scanner.Text()
if strings.Contains(s, sslcryp) || strings.Contains(s, passwords) {
t.Fatalf("Expected redacted MQSC output, got: %v", s)
}
}
err = scanner.Err()
if err != nil {
t.Fatal(err)
}
}
// TestRedactValidMQSC creates a new image with a Invalid MQSC file that contains sensitive information, starts a container based
// on that image, and checks that the MQSC has been redacted in the logs.
func TestRedactInvalidMQSC(t *testing.T) {
t.Parallel()
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
var buf bytes.Buffer
passwords := "hippoman4567"
sslcryp := fmt.Sprintf("GSK_PKCS11=/usr/lib/pkcs11/PKCS11_API.so;token-label;%s;SYMMETRIC_CIPHER_ON;", passwords)
/* LDAPPWD*/
fmt.Fprintf(&buf, "DEFINE AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) CONNAME('test(24)') SHORTUSR('sn') LDAPUSER('user') LDAPPWD('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPPPPPP('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD['%v']\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(ARGHHH) LDAPPWD('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) LDAPPWD('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) ARGHAHA(IDPWLDAP) LDAPPWD('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD '%v'\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD('%v') badvalues\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) badvales LDAPPWD('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD{'%v'}\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD<'%v'>\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD('%v'+\n p['il6])\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) AUTHTYPE(IDPWLDAP) LDAPPWD('%v'/653***)\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) LDAPPWD('%v'\n DISPLAY QMGR", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) LDAPPWD('%v💩')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) LDAPPWD💩('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) LDAP+\n 💩PWD('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) 💩 LDAPPWD('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) LDAPPWD 💩 ('%v')\n", passwords)
fmt.Fprintf(&buf, "ALTER AUTHINFO(TEST) LDAPPWD('%v') 💩\n", passwords)
fmt.Fprintf(&buf, "ALTER 💩 AUTHINFO(TEST) LDAPPWD('%v')\n", passwords)
var files = []struct {
Name, Body string
}{
{"Dockerfile", fmt.Sprintf(`
FROM %v
USER root
RUN rm -f /etc/mqm/*.mqsc
ADD test.mqsc /etc/mqm/
RUN chmod 0660 /etc/mqm/test.mqsc
USER mqm`, imageName())},
{"test.mqsc", buf.String()},
}
tag := createImage(t, cli, files)
defer deleteImage(t, cli, tag)
containerConfig := container.Config{
Env: []string{"LICENSE=accept", "MQ_QMGR_NAME=qm1"},
Image: tag,
}
id := runContainer(t, cli, &containerConfig)
defer cleanContainer(t, cli, id)
rc := waitForContainer(t, cli, id, 20*time.Second)
if rc != 1 {
t.Errorf("Expected rc=1, got rc=%v", rc)
}
scanner := bufio.NewScanner(strings.NewReader(inspectLogs(t, cli, id)))
for scanner.Scan() {
s := scanner.Text()
if strings.Contains(s, sslcryp) || strings.Contains(s, passwords) {
t.Fatalf("Expected redacted MQSC output, got: %v", s)
}
}
err = scanner.Err()
if err != nil {
t.Fatal(err)
}
}
// TestInvalidMQSC creates a new image with an MQSC file containing invalid MQSC,
// tries to start a container based on that image, and checks that container terminates
// func TestInvalidMQSC(t *testing.T) {
// t.Parallel()
// cli, err := client.NewEnvClient()
// if err != nil {
// t.Fatal(err)
// }
// var files = []struct {
// Name, Body string
// }{
// {"Dockerfile", fmt.Sprintf(`
// FROM %v
// USER root
// RUN rm -f /etc/mqm/*.mqsc
// ADD mqscTest.mqsc /etc/mqm/
// RUN chmod 0660 /etc/mqm/mqscTest.mqsc
// USER mqm`, imageName())},
// {"mqscTest.mqsc", "DEFINE INVALIDLISTENER('TEST.LISTENER.TCP') TRPTYPE(TCP) PORT(1414) CONTROL(QMGR) REPLACE"},
// }
// tag := createImage(t, cli, files)
// defer deleteImage(t, cli, tag)
func TestInvalidMQSC(t *testing.T) {
t.Parallel()
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
var files = []struct {
Name, Body string
}{
{"Dockerfile", fmt.Sprintf(`
FROM %v
USER root
RUN rm -f /etc/mqm/*.mqsc
ADD mqscTest.mqsc /etc/mqm/
RUN chmod 0660 /etc/mqm/mqscTest.mqsc
USER mqm`, imageName())},
{"mqscTest.mqsc", "DEFINE INVALIDLISTENER('TEST.LISTENER.TCP') TRPTYPE(TCP) PORT(1414) CONTROL(QMGR) REPLACE"},
}
tag := createImage(t, cli, files)
defer deleteImage(t, cli, tag)
// containerConfig := container.Config{
// Env: []string{"LICENSE=accept", "MQ_QMGR_NAME=qm1"},
// Image: tag,
// }
// id := runContainer(t, cli, &containerConfig)
// defer cleanContainer(t, cli, id)
// rc := waitForContainer(t, cli, id, 60*time.Second)
// if rc != 1 {
// t.Errorf("Expected rc=1, got rc=%v", rc)
// }
// expectTerminationMessage(t, cli, id)
// }
containerConfig := container.Config{
Env: []string{"LICENSE=accept", "MQ_QMGR_NAME=qm1"},
Image: tag,
}
id := runContainer(t, cli, &containerConfig)
defer cleanContainer(t, cli, id)
rc := waitForContainer(t, cli, id, 60*time.Second)
if rc != 1 {
t.Errorf("Expected rc=1, got rc=%v", rc)
}
expectTerminationMessage(t, cli, id)
}
// TestReadiness creates a new image with large amounts of MQSC in, to
// ensure that the readiness check doesn't pass until configuration has finished.
@@ -872,8 +1086,8 @@ func TestVersioning(t *testing.T) {
total := 6
foundCreated := false
foundRevision := false
foundSource := false
// foundRevision := false
// foundSource := false
foundMQVersion := false
foundMQLevel := false
foundMQLicense := false
@@ -894,30 +1108,30 @@ func TestVersioning(t *testing.T) {
}
}
if strings.Contains(line, "Image revision:") && !foundRevision {
total--
foundRevision = true
dataAr := strings.Split(line, " ")
data := dataAr[len(dataAr)-1]
// if strings.Contains(line, "Image revision:") && !foundRevision {
// total--
// foundRevision = true
// dataAr := strings.Split(line, " ")
// data := dataAr[len(dataAr)-1]
// Verify revision
pattern := regexp.MustCompile("^[a-fA-F0-9]{40}$")
if !pattern.MatchString(data) {
t.Errorf("Failed to validate revision (%v)", data)
}
}
// // Verify revision
// pattern := regexp.MustCompile("^[a-fA-F0-9]{40}$")
// if !pattern.MatchString(data) {
// t.Errorf("Failed to validate revision (%v)", data)
// }
// }
if strings.Contains(line, "Image source:") && !foundSource {
total--
foundSource = true
dataAr := strings.Split(line, " ")
data := dataAr[len(dataAr)-1]
// if strings.Contains(line, "Image source:") && !foundSource {
// total--
// foundSource = true
// dataAr := strings.Split(line, " ")
// data := dataAr[len(dataAr)-1]
// Verify source
if !strings.Contains(data, "github") {
t.Errorf("Failed to validate source (%v)", data)
}
}
// // Verify source
// if !strings.Contains(data, "github") {
// t.Errorf("Failed to validate source (%v)", data)
// }
// }
if strings.Contains(line, "MQ version:") && !foundMQVersion {
total--
@@ -963,7 +1177,11 @@ func TestVersioning(t *testing.T) {
}
}
if !foundCreated || !foundRevision || !foundSource || !foundMQVersion || !foundMQLevel || !foundMQLicense {
t.Errorf("Failed to find one or more version strings: created(%v) revision(%v) source(%v) mqversion(%v) mqlevel(%v) mqlicense(%v)", foundCreated, foundRevision, foundSource, foundMQVersion, foundMQLevel, foundMQLicense)
// if !foundCreated || !foundRevision || !foundSource || !foundMQVersion || !foundMQLevel || !foundMQLicense {
if !foundCreated || !foundMQVersion || !foundMQLevel || !foundMQLicense {
// t.Errorf("Failed to find one or more version strings: created(%v) revision(%v) source(%v) mqversion(%v) mqlevel(%v) mqlicense(%v)", foundCreated, foundRevision, foundSource, foundMQVersion, foundMQLevel, foundMQLicense)
t.Errorf("Failed to find one or more version strings: created(%v) mqversion(%v) mqlevel(%v) mqlicense(%v)", foundCreated, foundMQVersion, foundMQLevel, foundMQLicense)
}
}

View File

@@ -398,6 +398,14 @@ func stopContainer(t *testing.T, cli *client.Client, ID string) {
}
}
func killContainer(t *testing.T, cli *client.Client, ID string, signal string) {
t.Logf("Killing container: %v", ID)
err := cli.ContainerKill(context.Background(), ID, signal)
if err != nil {
t.Fatal(err)
}
}
func getExitCodeFilename(t *testing.T) string {
return t.Name() + "ExitCode"
}
@@ -522,6 +530,9 @@ func waitForReady(t *testing.T, cli *client.Client, ID string) {
if rc == 0 {
t.Log("MQ is ready")
return
} else if rc == 10 {
t.Log("MQ Readiness: Queue Manager Running as Standby")
return
}
case <-ctx.Done():
t.Fatal("Timed out waiting for container to become ready")
@@ -557,17 +568,17 @@ func removeNetwork(t *testing.T, cli *client.Client, ID string) {
}
}
func createVolume(t *testing.T, cli *client.Client) types.Volume {
func createVolume(t *testing.T, cli *client.Client, name string) types.Volume {
v, err := cli.VolumeCreate(context.Background(), volume.VolumesCreateBody{
Driver: "local",
DriverOpts: map[string]string{},
Labels: map[string]string{},
Name: t.Name(),
Name: name,
})
if err != nil {
t.Fatal(err)
}
t.Logf("Created volume %v", t.Name())
t.Logf("Created volume %v", v.Name)
return v
}

View File

@@ -0,0 +1,234 @@
/*
© Copyright IBM Corporation 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"strings"
"testing"
"time"
"github.com/docker/docker/client"
)
var miEnv = []string{
"LICENSE=accept",
"MQ_QMGR_NAME=QM1",
"MQ_MULTI_INSTANCE=true",
}
// TestMultiInstanceStartStop creates 2 containers in a multi instance queue manager configuration
// and starts/stop them checking we always have an active and standby
func TestMultiInstanceStartStop(t *testing.T) {
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
err, qm1aId, qm1bId, volumes := configureMultiInstance(t, cli)
if err != nil {
t.Fatal(err)
}
for _, volume := range volumes {
defer removeVolume(t, cli, volume)
}
defer cleanContainer(t, cli, qm1aId)
defer cleanContainer(t, cli, qm1bId)
waitForReady(t, cli, qm1aId)
waitForReady(t, cli, qm1bId)
err, active, standby := getActiveStandbyQueueManager(t, cli, qm1aId, qm1bId)
if err != nil {
t.Fatal(err)
}
killContainer(t, cli, active, "SIGTERM")
time.Sleep(2 * time.Second)
if status := getQueueManagerStatus(t, cli, standby, "QM1"); strings.Compare(status, "Running") != 0 {
t.Fatalf("Expected QM1 to be running as active queue manager, dspmq returned status of %v", status)
}
startContainer(t, cli, qm1aId)
waitForReady(t, cli, qm1aId)
err, _, _ = getActiveStandbyQueueManager(t, cli, qm1aId, qm1bId)
if err != nil {
t.Fatal(err)
}
}
// TestMultiInstanceContainerStop starts 2 containers in a multi instance queue manager configuration,
// stops the active queue manager, then checks to ensure the backup queue manager becomes active
func TestMultiInstanceContainerStop(t *testing.T) {
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
err, qm1aId, qm1bId, volumes := configureMultiInstance(t, cli)
if err != nil {
t.Fatal(err)
}
for _, volume := range volumes {
defer removeVolume(t, cli, volume)
}
defer cleanContainer(t, cli, qm1aId)
defer cleanContainer(t, cli, qm1bId)
waitForReady(t, cli, qm1aId)
waitForReady(t, cli, qm1bId)
err, active, standby := getActiveStandbyQueueManager(t, cli, qm1aId, qm1bId)
if err != nil {
t.Fatal(err)
}
stopContainer(t, cli, active)
if status := getQueueManagerStatus(t, cli, standby, "QM1"); strings.Compare(status, "Running") != 0 {
t.Fatalf("Expected QM1 to be running as active queue manager, dspmq returned status of %v", status)
}
}
// TestMultiInstanceRace starts 2 containers in separate goroutines in a multi instance queue manager
// configuration, then checks to ensure that both an active and standby queue manager have been started
func TestMultiInstanceRace(t *testing.T) {
t.Skipf("Skipping %v until file lock is implemented", t.Name())
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
qmsharedlogs := createVolume(t, cli, "qmsharedlogs")
defer removeVolume(t, cli, qmsharedlogs.Name)
qmshareddata := createVolume(t, cli, "qmshareddata")
defer removeVolume(t, cli, qmshareddata.Name)
qmsChannel := make(chan QMChan)
go singleMultiInstanceQueueManager(t, cli, qmsharedlogs.Name, qmshareddata.Name, qmsChannel)
go singleMultiInstanceQueueManager(t, cli, qmsharedlogs.Name, qmshareddata.Name, qmsChannel)
qm1a := <-qmsChannel
if qm1a.Error != nil {
t.Fatal(qm1a.Error)
}
qm1b := <-qmsChannel
if qm1b.Error != nil {
t.Fatal(qm1b.Error)
}
qm1aId, qm1aData := qm1a.QMId, qm1a.QMData
qm1bId, qm1bData := qm1b.QMId, qm1b.QMData
defer removeVolume(t, cli, qm1aData)
defer removeVolume(t, cli, qm1bData)
defer cleanContainer(t, cli, qm1aId)
defer cleanContainer(t, cli, qm1bId)
waitForReady(t, cli, qm1aId)
waitForReady(t, cli, qm1bId)
err, _, _ = getActiveStandbyQueueManager(t, cli, qm1aId, qm1bId)
if err != nil {
t.Fatal(err)
}
}
// TestMultiInstanceNoSharedMounts starts 2 multi instance queue managers without providing shared log/data
// mounts, then checks to ensure that the container terminates with the expected message
func TestMultiInstanceNoSharedMounts(t *testing.T) {
t.Parallel()
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
err, qm1aId, qm1aData := startMultiVolumeQueueManager(t, cli, true, "", "", miEnv)
if err != nil {
t.Fatal(err)
}
defer removeVolume(t, cli, qm1aData)
defer cleanContainer(t, cli, qm1aId)
waitForTerminationMessage(t, cli, qm1aId, "Missing required mount '/mnt/mqm-log'", 30*time.Second)
}
// TestMultiInstanceNoSharedLogs starts 2 multi instance queue managers without providing a shared log
// mount, then checks to ensure that the container terminates with the expected message
func TestMultiInstanceNoSharedLogs(t *testing.T) {
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
qmshareddata := createVolume(t, cli, "qmshareddata")
defer removeVolume(t, cli, qmshareddata.Name)
err, qm1aId, qm1aData := startMultiVolumeQueueManager(t, cli, true, "", qmshareddata.Name, miEnv)
if err != nil {
t.Fatal(err)
}
defer removeVolume(t, cli, qm1aData)
defer cleanContainer(t, cli, qm1aId)
waitForTerminationMessage(t, cli, qm1aId, "Missing required mount '/mnt/mqm-log'", 30*time.Second)
}
// TestMultiInstanceNoSharedData starts 2 multi instance queue managers without providing a shared data
// mount, then checks to ensure that the container terminates with the expected message
func TestMultiInstanceNoSharedData(t *testing.T) {
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
qmsharedlogs := createVolume(t, cli, "qmsharedlogs")
defer removeVolume(t, cli, qmsharedlogs.Name)
err, qm1aId, qm1aData := startMultiVolumeQueueManager(t, cli, true, qmsharedlogs.Name, "", miEnv)
if err != nil {
t.Fatal(err)
}
defer removeVolume(t, cli, qm1aData)
defer cleanContainer(t, cli, qm1aId)
waitForTerminationMessage(t, cli, qm1aId, "Missing required mount '/mnt/mqm-data'", 30*time.Second)
}
// TestMultiInstanceNoMounts starts 2 multi instance queue managers without providing a shared data
// mount, then checks to ensure that the container terminates with the expected message
func TestMultiInstanceNoMounts(t *testing.T) {
cli, err := client.NewEnvClient()
if err != nil {
t.Fatal(err)
}
err, qm1aId, qm1aData := startMultiVolumeQueueManager(t, cli, false, "", "", miEnv)
if err != nil {
t.Fatal(err)
}
defer removeVolume(t, cli, qm1aData)
defer cleanContainer(t, cli, qm1aId)
waitForTerminationMessage(t, cli, qm1aId, "Missing required mount '/mnt/mqm'", 30*time.Second)
}

View File

@@ -0,0 +1,178 @@
/*
© Copyright IBM Corporation 2019
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"context"
"fmt"
"regexp"
"strconv"
"strings"
"testing"
"time"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/network"
"github.com/docker/docker/client"
)
type QMChan struct {
QMId string
QMData string
Error error
}
// configureMultiInstance creates the volumes and containers required for basic testing
// of multi instance queue managers. Returns error, qm1a ID, qm1b ID, slice of volume names
func configureMultiInstance(t *testing.T, cli *client.Client) (error, string, string, []string) {
qmsharedlogs := createVolume(t, cli, "qmsharedlogs")
qmshareddata := createVolume(t, cli, "qmshareddata")
err, qm1aId, qm1aData := startMultiVolumeQueueManager(t, cli, true, qmsharedlogs.Name, qmshareddata.Name, miEnv)
if err != nil {
return err, "", "", []string{}
}
time.Sleep(10 * time.Second)
err, qm1bId, qm1bData := startMultiVolumeQueueManager(t, cli, true, qmsharedlogs.Name, qmshareddata.Name, miEnv)
if err != nil {
return err, "", "", []string{}
}
volumes := []string{qmsharedlogs.Name, qmshareddata.Name, qm1aData, qm1bData}
return nil, qm1aId, qm1bId, volumes
}
func singleMultiInstanceQueueManager(t *testing.T, cli *client.Client, qmsharedlogs string, qmshareddata string, qmsChannel chan QMChan) {
err, qmId, qmData := startMultiVolumeQueueManager(t, cli, true, qmsharedlogs, qmshareddata, miEnv)
if err != nil {
qmsChannel <- QMChan{Error: err}
}
qmsChannel <- QMChan{QMId: qmId, QMData: qmData}
}
func getHostConfig(t *testing.T, mounts int, qmsharedlogs string, qmshareddata string, qmdata string) container.HostConfig {
var hostConfig container.HostConfig
switch mounts {
case 1:
hostConfig = container.HostConfig{
Binds: []string{
coverageBind(t),
qmdata + ":/mnt/mqm",
},
}
case 2:
hostConfig = container.HostConfig{
Binds: []string{
coverageBind(t),
qmdata + ":/mnt/mqm",
qmshareddata + ":/mnt/mqm-data",
},
}
case 3:
hostConfig = container.HostConfig{
Binds: []string{
coverageBind(t),
qmdata + ":/mnt/mqm",
qmsharedlogs + ":/mnt/mqm-log",
},
}
case 4:
hostConfig = container.HostConfig{
Binds: []string{
coverageBind(t),
qmdata + ":/mnt/mqm",
qmsharedlogs + ":/mnt/mqm-log",
qmshareddata + ":/mnt/mqm-data",
},
}
}
return hostConfig
}
func startMultiVolumeQueueManager(t *testing.T, cli *client.Client, dataVol bool, qmsharedlogs string, qmshareddata string, env []string) (error, string, string) {
id := strconv.FormatInt(time.Now().UnixNano(), 10)
qmdata := createVolume(t, cli, id)
containerConfig := container.Config{
Image: imageName(),
Env: env,
}
var hostConfig container.HostConfig
if !dataVol {
hostConfig = container.HostConfig{}
} else if qmsharedlogs == "" && qmshareddata == "" {
hostConfig = getHostConfig(t, 1, "", "", qmdata.Name)
} else if qmsharedlogs == "" {
hostConfig = getHostConfig(t, 2, "", qmshareddata, qmdata.Name)
} else if qmshareddata == "" {
hostConfig = getHostConfig(t, 3, qmsharedlogs, "", qmdata.Name)
} else {
hostConfig = getHostConfig(t, 4, qmsharedlogs, qmshareddata, qmdata.Name)
}
networkingConfig := network.NetworkingConfig{}
qm, err := cli.ContainerCreate(context.Background(), &containerConfig, &hostConfig, &networkingConfig, t.Name()+id)
if err != nil {
return err, "", ""
}
startContainer(t, cli, qm.ID)
return nil, qm.ID, qmdata.Name
}
func getActiveStandbyQueueManager(t *testing.T, cli *client.Client, qm1aId string, qm1bId string) (error, string, string) {
qm1aStatus := getQueueManagerStatus(t, cli, qm1aId, "QM1")
qm1bStatus := getQueueManagerStatus(t, cli, qm1bId, "QM1")
if qm1aStatus == "Running" && qm1bStatus == "Running as standby" {
return nil, qm1aId, qm1bId
} else if qm1bStatus == "Running" && qm1aStatus == "Running as standby" {
return nil, qm1bId, qm1aId
}
err := fmt.Errorf("Expected to be running in multi instance configuration, got status 1) %v status 2) %v", qm1aStatus, qm1bStatus)
return err, "", ""
}
func getQueueManagerStatus(t *testing.T, cli *client.Client, containerID string, queueManagerName string) string {
_, dspmqOut := execContainer(t, cli, containerID, "mqm", []string{"bash", "-c", "dspmq", "-m", queueManagerName})
regex := regexp.MustCompile(`STATUS\(.*\)`)
status := regex.FindString(dspmqOut)
status = strings.TrimSuffix(strings.TrimPrefix(status, "STATUS("), ")")
return status
}
func waitForTerminationMessage(t *testing.T, cli *client.Client, qmId string, terminationString string, timeout time.Duration) {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
for {
select {
case <-time.After(1 * time.Second):
m := terminationMessage(t, cli, qmId)
if m != "" {
if !strings.Contains(m, terminationString) {
t.Fatalf("Expected container to fail on missing required mount. Got termination message: %v", m)
}
return
}
case <-ctx.Done():
t.Fatal("Timed out waiting for container to terminate")
}
}
}

View File

@@ -1,4 +1,4 @@
# © Copyright IBM Corporation 2018
# © Copyright IBM Corporation 2018, 2019
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@@ -15,7 +15,7 @@
###############################################################################
# Application build environment (Maven)
###############################################################################
FROM maven:3-ibmjava as builder
FROM docker.io/maven:3-ibmjava as builder
COPY pom.xml /usr/src/mymaven/
WORKDIR /usr/src/mymaven
# Download dependencies separately, so Docker caches them
@@ -30,7 +30,7 @@ RUN find /usr/src/mymaven
###############################################################################
# Application runtime (JRE only, no build environment)
###############################################################################
FROM ibmjava:8-jre
FROM docker.io/ibmjava:8-jre
COPY --from=builder /usr/src/mymaven/target/*.jar /opt/app/
COPY --from=builder /usr/src/mymaven/target/lib/*.jar /opt/app/
ENTRYPOINT ["java", "-classpath", "/opt/app/*", "org.junit.platform.console.ConsoleLauncher", "-p", "com.ibm.mqcontainer.test", "--details", "verbose"]

View File

@@ -73,7 +73,7 @@ rm -rf ${mnt_mq}/etc/yum.repos.d/*
buildah config \
--os linux \
--label architecture=x86_64 \
--label architecture=amd64 \
--label name="${imagename%:*}" \
--cmd "" \
--entrypoint '["java", "-classpath", "/opt/app/*", "org.junit.platform.console.ConsoleLauncher", "-p", "com.ibm.mqcontainer.test", "--details", "verbose"]' \

View File

@@ -0,0 +1,10 @@
# Treat all files in this repo as binary, with no git magic updating
# line endings. Windows users contributing to Go will need to use a
# modern version of git and editors capable of LF line endings.
#
# We'll prevent accidental CRLF line endings from entering the repo
# via the git-review gofmt checks.
#
# See golang.org/issue/9281
* -text

2
vendor/software.sslmate.com/src/go-pkcs12/.gitignore generated vendored Normal file
View File

@@ -0,0 +1,2 @@
# Add no patterns to .hgignore except for files generated by the build.
last-change

28
vendor/software.sslmate.com/src/go-pkcs12/LICENSE generated vendored Normal file
View File

@@ -0,0 +1,28 @@
Copyright (c) 2015, 2018, 2019 Opsmate, Inc. All rights reserved.
Copyright (c) 2009 The Go Authors. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

35
vendor/software.sslmate.com/src/go-pkcs12/README.md generated vendored Normal file
View File

@@ -0,0 +1,35 @@
# package pkcs12
[![GoDoc](https://godoc.org/software.sslmate.com/src/go-pkcs12?status.svg)](https://godoc.org/software.sslmate.com/src/go-pkcs12)
import "software.sslmate.com/src/go-pkcs12"
Package pkcs12 implements some of PKCS#12 (also known as P12 or PFX).
It is intended for decoding P12/PFX files for use with the `crypto/tls`
package, and for encoding P12/PFX files for use by legacy applications which
do not support newer formats. Since PKCS#12 uses weak encryption
primitives, it SHOULD NOT be used for new applications.
This package is forked from `golang.org/x/crypto/pkcs12`, which is frozen.
The implementation is distilled from https://tools.ietf.org/html/rfc7292
and referenced documents.
This repository holds supplementary Go cryptography libraries.
## Import Path
Note that although the source code and issue tracker for this package are hosted
on GitHub, the import path is:
software.sslmate.com/src/go-pkcs12
Please be sure to use this path when you `go get` and `import` this package.
## Download/Install
The easiest way to install is to run `go get -u software.sslmate.com/src/go-pkcs12`. You
can also manually git clone the repository to `$GOPATH/src/software.sslmate.com/src/go-pkcs12`.
## Report Issues / Send Patches
Open an issue or PR at https://github.com/SSLMate/go-pkcs12

View File

@@ -0,0 +1,50 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package pkcs12
import (
"errors"
"unicode/utf16"
)
// bmpString returns s encoded in UCS-2 with a zero terminator.
func bmpString(s string) ([]byte, error) {
// References:
// https://tools.ietf.org/html/rfc7292#appendix-B.1
// https://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_Multilingual_Plane
// - non-BMP characters are encoded in UTF 16 by using a surrogate pair of 16-bit codes
// EncodeRune returns 0xfffd if the rune does not need special encoding
// - the above RFC provides the info that BMPStrings are NULL terminated.
ret := make([]byte, 0, 2*len(s)+2)
for _, r := range s {
if t, _ := utf16.EncodeRune(r); t != 0xfffd {
return nil, errors.New("pkcs12: string contains characters that cannot be encoded in UCS-2")
}
ret = append(ret, byte(r/256), byte(r%256))
}
return append(ret, 0, 0), nil
}
func decodeBMPString(bmpString []byte) (string, error) {
if len(bmpString)%2 != 0 {
return "", errors.New("pkcs12: odd-length BMP string")
}
// strip terminator if present
if l := len(bmpString); l >= 2 && bmpString[l-1] == 0 && bmpString[l-2] == 0 {
bmpString = bmpString[:l-2]
}
s := make([]uint16, 0, len(bmpString)/2)
for len(bmpString) > 0 {
s = append(s, uint16(bmpString[0])<<8+uint16(bmpString[1]))
bmpString = bmpString[2:]
}
return string(utf16.Decode(s)), nil
}

View File

@@ -0,0 +1,63 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package pkcs12
import (
"bytes"
"encoding/hex"
"testing"
)
var bmpStringTests = []struct {
in string
expectedHex string
shouldFail bool
}{
{"", "0000", false},
// Example from https://tools.ietf.org/html/rfc7292#appendix-B.
{"Beavis", "0042006500610076006900730000", false},
// Some characters from the "Letterlike Symbols Unicode block".
{"\u2115 - Double-struck N", "21150020002d00200044006f00750062006c0065002d00730074007200750063006b0020004e0000", false},
// any character outside the BMP should trigger an error.
{"\U0001f000 East wind (Mahjong)", "", true},
}
func TestBMPString(t *testing.T) {
for i, test := range bmpStringTests {
expected, err := hex.DecodeString(test.expectedHex)
if err != nil {
t.Fatalf("#%d: failed to decode expectation", i)
}
out, err := bmpString(test.in)
if err == nil && test.shouldFail {
t.Errorf("#%d: expected to fail, but produced %x", i, out)
continue
}
if err != nil && !test.shouldFail {
t.Errorf("#%d: failed unexpectedly: %s", i, err)
continue
}
if !test.shouldFail {
if !bytes.Equal(out, expected) {
t.Errorf("#%d: expected %s, got %x", i, test.expectedHex, out)
continue
}
roundTrip, err := decodeBMPString(out)
if err != nil {
t.Errorf("#%d: decoding output gave an error: %s", i, err)
continue
}
if roundTrip != test.in {
t.Errorf("#%d: decoding output resulted in %q, but it should have been %q", i, roundTrip, test.in)
continue
}
}
}
}

173
vendor/software.sslmate.com/src/go-pkcs12/crypto.go generated vendored Normal file
View File

@@ -0,0 +1,173 @@
// Copyright 2015, 2018, 2019 Opsmate, Inc. All rights reserved.
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package pkcs12
import (
"bytes"
"crypto/cipher"
"crypto/des"
"crypto/x509/pkix"
"encoding/asn1"
"errors"
"software.sslmate.com/src/go-pkcs12/internal/rc2"
)
var (
oidPBEWithSHAAnd3KeyTripleDESCBC = asn1.ObjectIdentifier([]int{1, 2, 840, 113549, 1, 12, 1, 3})
oidPBEWithSHAAnd40BitRC2CBC = asn1.ObjectIdentifier([]int{1, 2, 840, 113549, 1, 12, 1, 6})
)
// pbeCipher is an abstraction of a PKCS#12 cipher.
type pbeCipher interface {
// create returns a cipher.Block given a key.
create(key []byte) (cipher.Block, error)
// deriveKey returns a key derived from the given password and salt.
deriveKey(salt, password []byte, iterations int) []byte
// deriveKey returns an IV derived from the given password and salt.
deriveIV(salt, password []byte, iterations int) []byte
}
type shaWithTripleDESCBC struct{}
func (shaWithTripleDESCBC) create(key []byte) (cipher.Block, error) {
return des.NewTripleDESCipher(key)
}
func (shaWithTripleDESCBC) deriveKey(salt, password []byte, iterations int) []byte {
return pbkdf(sha1Sum, 20, 64, salt, password, iterations, 1, 24)
}
func (shaWithTripleDESCBC) deriveIV(salt, password []byte, iterations int) []byte {
return pbkdf(sha1Sum, 20, 64, salt, password, iterations, 2, 8)
}
type shaWith40BitRC2CBC struct{}
func (shaWith40BitRC2CBC) create(key []byte) (cipher.Block, error) {
return rc2.New(key, len(key)*8)
}
func (shaWith40BitRC2CBC) deriveKey(salt, password []byte, iterations int) []byte {
return pbkdf(sha1Sum, 20, 64, salt, password, iterations, 1, 5)
}
func (shaWith40BitRC2CBC) deriveIV(salt, password []byte, iterations int) []byte {
return pbkdf(sha1Sum, 20, 64, salt, password, iterations, 2, 8)
}
type pbeParams struct {
Salt []byte
Iterations int
}
func pbeCipherFor(algorithm pkix.AlgorithmIdentifier, password []byte) (cipher.Block, []byte, error) {
var cipherType pbeCipher
switch {
case algorithm.Algorithm.Equal(oidPBEWithSHAAnd3KeyTripleDESCBC):
cipherType = shaWithTripleDESCBC{}
case algorithm.Algorithm.Equal(oidPBEWithSHAAnd40BitRC2CBC):
cipherType = shaWith40BitRC2CBC{}
default:
return nil, nil, NotImplementedError("algorithm " + algorithm.Algorithm.String() + " is not supported")
}
var params pbeParams
if err := unmarshal(algorithm.Parameters.FullBytes, &params); err != nil {
return nil, nil, err
}
key := cipherType.deriveKey(params.Salt, password, params.Iterations)
iv := cipherType.deriveIV(params.Salt, password, params.Iterations)
block, err := cipherType.create(key)
if err != nil {
return nil, nil, err
}
return block, iv, nil
}
func pbDecrypterFor(algorithm pkix.AlgorithmIdentifier, password []byte) (cipher.BlockMode, int, error) {
block, iv, err := pbeCipherFor(algorithm, password)
if err != nil {
return nil, 0, err
}
return cipher.NewCBCDecrypter(block, iv), block.BlockSize(), nil
}
func pbDecrypt(info decryptable, password []byte) (decrypted []byte, err error) {
cbc, blockSize, err := pbDecrypterFor(info.Algorithm(), password)
if err != nil {
return nil, err
}
encrypted := info.Data()
if len(encrypted) == 0 {
return nil, errors.New("pkcs12: empty encrypted data")
}
if len(encrypted)%blockSize != 0 {
return nil, errors.New("pkcs12: input is not a multiple of the block size")
}
decrypted = make([]byte, len(encrypted))
cbc.CryptBlocks(decrypted, encrypted)
psLen := int(decrypted[len(decrypted)-1])
if psLen == 0 || psLen > blockSize {
return nil, ErrDecryption
}
if len(decrypted) < psLen {
return nil, ErrDecryption
}
ps := decrypted[len(decrypted)-psLen:]
decrypted = decrypted[:len(decrypted)-psLen]
if bytes.Compare(ps, bytes.Repeat([]byte{byte(psLen)}, psLen)) != 0 {
return nil, ErrDecryption
}
return
}
// decryptable abstracts an object that contains ciphertext.
type decryptable interface {
Algorithm() pkix.AlgorithmIdentifier
Data() []byte
}
func pbEncrypterFor(algorithm pkix.AlgorithmIdentifier, password []byte) (cipher.BlockMode, int, error) {
block, iv, err := pbeCipherFor(algorithm, password)
if err != nil {
return nil, 0, err
}
return cipher.NewCBCEncrypter(block, iv), block.BlockSize(), nil
}
func pbEncrypt(info encryptable, decrypted []byte, password []byte) error {
cbc, blockSize, err := pbEncrypterFor(info.Algorithm(), password)
if err != nil {
return err
}
psLen := blockSize - len(decrypted)%blockSize
encrypted := make([]byte, len(decrypted)+psLen)
copy(encrypted[:len(decrypted)], decrypted)
copy(encrypted[len(decrypted):], bytes.Repeat([]byte{byte(psLen)}, psLen))
cbc.CryptBlocks(encrypted, encrypted)
info.SetData(encrypted)
return nil
}
// encryptable abstracts a object that contains ciphertext.
type encryptable interface {
Algorithm() pkix.AlgorithmIdentifier
SetData([]byte)
}

View File

@@ -0,0 +1,194 @@
// Copyright 2015, 2018, 2019 Opsmate, Inc. All rights reserved.
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package pkcs12
import (
"bytes"
"crypto/x509/pkix"
"encoding/asn1"
"testing"
)
var sha1WithTripleDES = asn1.ObjectIdentifier([]int{1, 2, 840, 113549, 1, 12, 1, 3})
func TestPbDecrypterFor(t *testing.T) {
params, _ := asn1.Marshal(pbeParams{
Salt: []byte{1, 2, 3, 4, 5, 6, 7, 8},
Iterations: 2048,
})
alg := pkix.AlgorithmIdentifier{
Algorithm: asn1.ObjectIdentifier([]int{1, 2, 3}),
Parameters: asn1.RawValue{
FullBytes: params,
},
}
pass, _ := bmpString("Sesame open")
_, _, err := pbDecrypterFor(alg, pass)
if _, ok := err.(NotImplementedError); !ok {
t.Errorf("expected not implemented error, got: %T %s", err, err)
}
alg.Algorithm = sha1WithTripleDES
cbc, blockSize, err := pbDecrypterFor(alg, pass)
if err != nil {
t.Errorf("unexpected error from pbDecrypterFor %v", err)
}
if blockSize != 8 {
t.Errorf("unexpected block size %d, wanted 8", blockSize)
}
plaintext := []byte{1, 2, 3, 4, 5, 6, 7, 8}
expectedCiphertext := []byte{185, 73, 135, 249, 137, 1, 122, 247}
ciphertext := make([]byte, len(plaintext))
cbc.CryptBlocks(ciphertext, plaintext)
if bytes.Compare(ciphertext, expectedCiphertext) != 0 {
t.Errorf("bad ciphertext, got %x but wanted %x", ciphertext, expectedCiphertext)
}
}
func TestPbEncrypterFor(t *testing.T) {
params, _ := asn1.Marshal(pbeParams{
Salt: []byte{1, 2, 3, 4, 5, 6, 7, 8},
Iterations: 2048,
})
alg := pkix.AlgorithmIdentifier{
Algorithm: asn1.ObjectIdentifier([]int{1, 2, 3}),
Parameters: asn1.RawValue{
FullBytes: params,
},
}
pass, _ := bmpString("Sesame open")
_, _, err := pbEncrypterFor(alg, pass)
if _, ok := err.(NotImplementedError); !ok {
t.Errorf("expected not implemented error, got: %T %s", err, err)
}
alg.Algorithm = asn1.ObjectIdentifier([]int{1, 2, 840, 113549, 1, 12, 1, 3})
cbc, _, err := pbEncrypterFor(alg, pass)
if err != nil {
t.Errorf("err: %v", err)
}
expectedM := []byte{1, 2, 3, 4, 5, 6, 7, 8}
M := []byte{185, 73, 135, 249, 137, 1, 122, 247}
cbc.CryptBlocks(M, M)
if bytes.Compare(M, expectedM) != 0 {
t.Errorf("expected M to be '%d', but found '%d", expectedM, M)
}
}
var pbDecryptTests = []struct {
in []byte
expected []byte
expectedError error
}{
{
[]byte("\x33\x73\xf3\x9f\xda\x49\xae\xfc\xa0\x9a\xdf\x5a\x58\xa0\xea\x46"), // 7 padding bytes
[]byte("A secret!"),
nil,
},
{
[]byte("\x33\x73\xf3\x9f\xda\x49\xae\xfc\x96\x24\x2f\x71\x7e\x32\x3f\xe7"), // 8 padding bytes
[]byte("A secret"),
nil,
},
{
[]byte("\x35\x0c\xc0\x8d\xab\xa9\x5d\x30\x7f\x9a\xec\x6a\xd8\x9b\x9c\xd9"), // 9 padding bytes, incorrect
nil,
ErrDecryption,
},
{
[]byte("\xb2\xf9\x6e\x06\x60\xae\x20\xcf\x08\xa0\x7b\xd9\x6b\x20\xef\x41"), // incorrect padding bytes: [ ... 0x04 0x02 ]
nil,
ErrDecryption,
},
}
func TestPbDecrypt(t *testing.T) {
for i, test := range pbDecryptTests {
decryptable := testDecryptable{
data: test.in,
algorithm: pkix.AlgorithmIdentifier{
Algorithm: sha1WithTripleDES,
Parameters: pbeParams{
Salt: []byte("\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8"),
Iterations: 4096,
}.RawASN1(),
},
}
password, _ := bmpString("sesame")
plaintext, err := pbDecrypt(decryptable, password)
if err != test.expectedError {
t.Errorf("#%d: got error %q, but wanted %q", i, err, test.expectedError)
continue
}
if !bytes.Equal(plaintext, test.expected) {
t.Errorf("#%d: got %x, but wanted %x", i, plaintext, test.expected)
}
}
}
func TestPbEncrypt(t *testing.T) {
tests := [][]byte{
[]byte("A secret!"),
[]byte("A secret"),
}
expected := [][]byte{
[]byte("\x33\x73\xf3\x9f\xda\x49\xae\xfc\xa0\x9a\xdf\x5a\x58\xa0\xea\x46"), // 7 padding bytes
[]byte("\x33\x73\xf3\x9f\xda\x49\xae\xfc\x96\x24\x2f\x71\x7e\x32\x3f\xe7"), // 8 padding bytes
}
for i, c := range tests {
td := testDecryptable{
algorithm: pkix.AlgorithmIdentifier{
Algorithm: asn1.ObjectIdentifier([]int{1, 2, 840, 113549, 1, 12, 1, 3}), // SHA1/3TDES
Parameters: pbeParams{
Salt: []byte("\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8"),
Iterations: 4096,
}.RawASN1(),
},
}
p, _ := bmpString("sesame")
err := pbEncrypt(&td, c, p)
if err != nil {
t.Errorf("error encrypting %d: %v", c, err)
}
if bytes.Compare(td.data, expected[i]) != 0 {
t.Errorf("expected %d to be encrypted to %d, but found %d", c, expected[i], td.data)
}
}
}
type testDecryptable struct {
data []byte
algorithm pkix.AlgorithmIdentifier
}
func (d testDecryptable) Algorithm() pkix.AlgorithmIdentifier { return d.algorithm }
func (d testDecryptable) Data() []byte { return d.data }
func (d *testDecryptable) SetData(data []byte) { d.data = data }
func (params pbeParams) RawASN1() (raw asn1.RawValue) {
asn1Bytes, err := asn1.Marshal(params)
if err != nil {
panic(err)
}
_, err = asn1.Unmarshal(asn1Bytes, &raw)
if err != nil {
panic(err)
}
return
}

23
vendor/software.sslmate.com/src/go-pkcs12/errors.go generated vendored Normal file
View File

@@ -0,0 +1,23 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package pkcs12
import "errors"
var (
// ErrDecryption represents a failure to decrypt the input.
ErrDecryption = errors.New("pkcs12: decryption error, incorrect padding")
// ErrIncorrectPassword is returned when an incorrect password is detected.
// Usually, P12/PFX data is signed to be able to verify the password.
ErrIncorrectPassword = errors.New("pkcs12: decryption password incorrect")
)
// NotImplementedError indicates that the input is not currently supported.
type NotImplementedError string
func (e NotImplementedError) Error() string {
return "pkcs12: " + string(e)
}

View File

@@ -0,0 +1,27 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package rc2
import (
"testing"
)
func BenchmarkEncrypt(b *testing.B) {
r, _ := New([]byte{0, 0, 0, 0, 0, 0, 0, 0}, 64)
b.ResetTimer()
var src [8]byte
for i := 0; i < b.N; i++ {
r.Encrypt(src[:], src[:])
}
}
func BenchmarkDecrypt(b *testing.B) {
r, _ := New([]byte{0, 0, 0, 0, 0, 0, 0, 0}, 64)
b.ResetTimer()
var src [8]byte
for i := 0; i < b.N; i++ {
r.Decrypt(src[:], src[:])
}
}

View File

@@ -0,0 +1,271 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package rc2 implements the RC2 cipher
/*
https://www.ietf.org/rfc/rfc2268.txt
http://people.csail.mit.edu/rivest/pubs/KRRR98.pdf
This code is licensed under the MIT license.
*/
package rc2
import (
"crypto/cipher"
"encoding/binary"
)
// The rc2 block size in bytes
const BlockSize = 8
type rc2Cipher struct {
k [64]uint16
}
// New returns a new rc2 cipher with the given key and effective key length t1
func New(key []byte, t1 int) (cipher.Block, error) {
// TODO(dgryski): error checking for key length
return &rc2Cipher{
k: expandKey(key, t1),
}, nil
}
func (*rc2Cipher) BlockSize() int { return BlockSize }
var piTable = [256]byte{
0xd9, 0x78, 0xf9, 0xc4, 0x19, 0xdd, 0xb5, 0xed, 0x28, 0xe9, 0xfd, 0x79, 0x4a, 0xa0, 0xd8, 0x9d,
0xc6, 0x7e, 0x37, 0x83, 0x2b, 0x76, 0x53, 0x8e, 0x62, 0x4c, 0x64, 0x88, 0x44, 0x8b, 0xfb, 0xa2,
0x17, 0x9a, 0x59, 0xf5, 0x87, 0xb3, 0x4f, 0x13, 0x61, 0x45, 0x6d, 0x8d, 0x09, 0x81, 0x7d, 0x32,
0xbd, 0x8f, 0x40, 0xeb, 0x86, 0xb7, 0x7b, 0x0b, 0xf0, 0x95, 0x21, 0x22, 0x5c, 0x6b, 0x4e, 0x82,
0x54, 0xd6, 0x65, 0x93, 0xce, 0x60, 0xb2, 0x1c, 0x73, 0x56, 0xc0, 0x14, 0xa7, 0x8c, 0xf1, 0xdc,
0x12, 0x75, 0xca, 0x1f, 0x3b, 0xbe, 0xe4, 0xd1, 0x42, 0x3d, 0xd4, 0x30, 0xa3, 0x3c, 0xb6, 0x26,
0x6f, 0xbf, 0x0e, 0xda, 0x46, 0x69, 0x07, 0x57, 0x27, 0xf2, 0x1d, 0x9b, 0xbc, 0x94, 0x43, 0x03,
0xf8, 0x11, 0xc7, 0xf6, 0x90, 0xef, 0x3e, 0xe7, 0x06, 0xc3, 0xd5, 0x2f, 0xc8, 0x66, 0x1e, 0xd7,
0x08, 0xe8, 0xea, 0xde, 0x80, 0x52, 0xee, 0xf7, 0x84, 0xaa, 0x72, 0xac, 0x35, 0x4d, 0x6a, 0x2a,
0x96, 0x1a, 0xd2, 0x71, 0x5a, 0x15, 0x49, 0x74, 0x4b, 0x9f, 0xd0, 0x5e, 0x04, 0x18, 0xa4, 0xec,
0xc2, 0xe0, 0x41, 0x6e, 0x0f, 0x51, 0xcb, 0xcc, 0x24, 0x91, 0xaf, 0x50, 0xa1, 0xf4, 0x70, 0x39,
0x99, 0x7c, 0x3a, 0x85, 0x23, 0xb8, 0xb4, 0x7a, 0xfc, 0x02, 0x36, 0x5b, 0x25, 0x55, 0x97, 0x31,
0x2d, 0x5d, 0xfa, 0x98, 0xe3, 0x8a, 0x92, 0xae, 0x05, 0xdf, 0x29, 0x10, 0x67, 0x6c, 0xba, 0xc9,
0xd3, 0x00, 0xe6, 0xcf, 0xe1, 0x9e, 0xa8, 0x2c, 0x63, 0x16, 0x01, 0x3f, 0x58, 0xe2, 0x89, 0xa9,
0x0d, 0x38, 0x34, 0x1b, 0xab, 0x33, 0xff, 0xb0, 0xbb, 0x48, 0x0c, 0x5f, 0xb9, 0xb1, 0xcd, 0x2e,
0xc5, 0xf3, 0xdb, 0x47, 0xe5, 0xa5, 0x9c, 0x77, 0x0a, 0xa6, 0x20, 0x68, 0xfe, 0x7f, 0xc1, 0xad,
}
func expandKey(key []byte, t1 int) [64]uint16 {
l := make([]byte, 128)
copy(l, key)
var t = len(key)
var t8 = (t1 + 7) / 8
var tm = byte(255 % uint(1<<(8+uint(t1)-8*uint(t8))))
for i := len(key); i < 128; i++ {
l[i] = piTable[l[i-1]+l[uint8(i-t)]]
}
l[128-t8] = piTable[l[128-t8]&tm]
for i := 127 - t8; i >= 0; i-- {
l[i] = piTable[l[i+1]^l[i+t8]]
}
var k [64]uint16
for i := range k {
k[i] = uint16(l[2*i]) + uint16(l[2*i+1])*256
}
return k
}
func rotl16(x uint16, b uint) uint16 {
return (x >> (16 - b)) | (x << b)
}
func (c *rc2Cipher) Encrypt(dst, src []byte) {
r0 := binary.LittleEndian.Uint16(src[0:])
r1 := binary.LittleEndian.Uint16(src[2:])
r2 := binary.LittleEndian.Uint16(src[4:])
r3 := binary.LittleEndian.Uint16(src[6:])
var j int
for j <= 16 {
// mix r0
r0 = r0 + c.k[j] + (r3 & r2) + ((^r3) & r1)
r0 = rotl16(r0, 1)
j++
// mix r1
r1 = r1 + c.k[j] + (r0 & r3) + ((^r0) & r2)
r1 = rotl16(r1, 2)
j++
// mix r2
r2 = r2 + c.k[j] + (r1 & r0) + ((^r1) & r3)
r2 = rotl16(r2, 3)
j++
// mix r3
r3 = r3 + c.k[j] + (r2 & r1) + ((^r2) & r0)
r3 = rotl16(r3, 5)
j++
}
r0 = r0 + c.k[r3&63]
r1 = r1 + c.k[r0&63]
r2 = r2 + c.k[r1&63]
r3 = r3 + c.k[r2&63]
for j <= 40 {
// mix r0
r0 = r0 + c.k[j] + (r3 & r2) + ((^r3) & r1)
r0 = rotl16(r0, 1)
j++
// mix r1
r1 = r1 + c.k[j] + (r0 & r3) + ((^r0) & r2)
r1 = rotl16(r1, 2)
j++
// mix r2
r2 = r2 + c.k[j] + (r1 & r0) + ((^r1) & r3)
r2 = rotl16(r2, 3)
j++
// mix r3
r3 = r3 + c.k[j] + (r2 & r1) + ((^r2) & r0)
r3 = rotl16(r3, 5)
j++
}
r0 = r0 + c.k[r3&63]
r1 = r1 + c.k[r0&63]
r2 = r2 + c.k[r1&63]
r3 = r3 + c.k[r2&63]
for j <= 60 {
// mix r0
r0 = r0 + c.k[j] + (r3 & r2) + ((^r3) & r1)
r0 = rotl16(r0, 1)
j++
// mix r1
r1 = r1 + c.k[j] + (r0 & r3) + ((^r0) & r2)
r1 = rotl16(r1, 2)
j++
// mix r2
r2 = r2 + c.k[j] + (r1 & r0) + ((^r1) & r3)
r2 = rotl16(r2, 3)
j++
// mix r3
r3 = r3 + c.k[j] + (r2 & r1) + ((^r2) & r0)
r3 = rotl16(r3, 5)
j++
}
binary.LittleEndian.PutUint16(dst[0:], r0)
binary.LittleEndian.PutUint16(dst[2:], r1)
binary.LittleEndian.PutUint16(dst[4:], r2)
binary.LittleEndian.PutUint16(dst[6:], r3)
}
func (c *rc2Cipher) Decrypt(dst, src []byte) {
r0 := binary.LittleEndian.Uint16(src[0:])
r1 := binary.LittleEndian.Uint16(src[2:])
r2 := binary.LittleEndian.Uint16(src[4:])
r3 := binary.LittleEndian.Uint16(src[6:])
j := 63
for j >= 44 {
// unmix r3
r3 = rotl16(r3, 16-5)
r3 = r3 - c.k[j] - (r2 & r1) - ((^r2) & r0)
j--
// unmix r2
r2 = rotl16(r2, 16-3)
r2 = r2 - c.k[j] - (r1 & r0) - ((^r1) & r3)
j--
// unmix r1
r1 = rotl16(r1, 16-2)
r1 = r1 - c.k[j] - (r0 & r3) - ((^r0) & r2)
j--
// unmix r0
r0 = rotl16(r0, 16-1)
r0 = r0 - c.k[j] - (r3 & r2) - ((^r3) & r1)
j--
}
r3 = r3 - c.k[r2&63]
r2 = r2 - c.k[r1&63]
r1 = r1 - c.k[r0&63]
r0 = r0 - c.k[r3&63]
for j >= 20 {
// unmix r3
r3 = rotl16(r3, 16-5)
r3 = r3 - c.k[j] - (r2 & r1) - ((^r2) & r0)
j--
// unmix r2
r2 = rotl16(r2, 16-3)
r2 = r2 - c.k[j] - (r1 & r0) - ((^r1) & r3)
j--
// unmix r1
r1 = rotl16(r1, 16-2)
r1 = r1 - c.k[j] - (r0 & r3) - ((^r0) & r2)
j--
// unmix r0
r0 = rotl16(r0, 16-1)
r0 = r0 - c.k[j] - (r3 & r2) - ((^r3) & r1)
j--
}
r3 = r3 - c.k[r2&63]
r2 = r2 - c.k[r1&63]
r1 = r1 - c.k[r0&63]
r0 = r0 - c.k[r3&63]
for j >= 0 {
// unmix r3
r3 = rotl16(r3, 16-5)
r3 = r3 - c.k[j] - (r2 & r1) - ((^r2) & r0)
j--
// unmix r2
r2 = rotl16(r2, 16-3)
r2 = r2 - c.k[j] - (r1 & r0) - ((^r1) & r3)
j--
// unmix r1
r1 = rotl16(r1, 16-2)
r1 = r1 - c.k[j] - (r0 & r3) - ((^r0) & r2)
j--
// unmix r0
r0 = rotl16(r0, 16-1)
r0 = r0 - c.k[j] - (r3 & r2) - ((^r3) & r1)
j--
}
binary.LittleEndian.PutUint16(dst[0:], r0)
binary.LittleEndian.PutUint16(dst[2:], r1)
binary.LittleEndian.PutUint16(dst[4:], r2)
binary.LittleEndian.PutUint16(dst[6:], r3)
}

View File

@@ -0,0 +1,92 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package rc2
import (
"bytes"
"encoding/hex"
"testing"
)
func TestEncryptDecrypt(t *testing.T) {
// TODO(dgryski): add the rest of the test vectors from the RFC
var tests = []struct {
key string
plain string
cipher string
t1 int
}{
{
"0000000000000000",
"0000000000000000",
"ebb773f993278eff",
63,
},
{
"ffffffffffffffff",
"ffffffffffffffff",
"278b27e42e2f0d49",
64,
},
{
"3000000000000000",
"1000000000000001",
"30649edf9be7d2c2",
64,
},
{
"88",
"0000000000000000",
"61a8a244adacccf0",
64,
},
{
"88bca90e90875a",
"0000000000000000",
"6ccf4308974c267f",
64,
},
{
"88bca90e90875a7f0f79c384627bafb2",
"0000000000000000",
"1a807d272bbe5db1",
64,
},
{
"88bca90e90875a7f0f79c384627bafb2",
"0000000000000000",
"2269552ab0f85ca6",
128,
},
{
"88bca90e90875a7f0f79c384627bafb216f80a6f85920584c42fceb0be255daf1e",
"0000000000000000",
"5b78d3a43dfff1f1",
129,
},
}
for _, tt := range tests {
k, _ := hex.DecodeString(tt.key)
p, _ := hex.DecodeString(tt.plain)
c, _ := hex.DecodeString(tt.cipher)
b, _ := New(k, tt.t1)
var dst [8]byte
b.Encrypt(dst[:], p)
if !bytes.Equal(dst[:], c) {
t.Errorf("encrypt failed: got % 2x wanted % 2x\n", dst, c)
}
b.Decrypt(dst[:], c)
if !bytes.Equal(dst[:], p) {
t.Errorf("decrypt failed: got % 2x wanted % 2x\n", dst, p)
}
}
}

60
vendor/software.sslmate.com/src/go-pkcs12/mac.go generated vendored Normal file
View File

@@ -0,0 +1,60 @@
// Copyright 2015, 2018, 2019 Opsmate, Inc. All rights reserved.
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package pkcs12
import (
"crypto/hmac"
"crypto/sha1"
"crypto/x509/pkix"
"encoding/asn1"
)
type macData struct {
Mac digestInfo
MacSalt []byte
Iterations int `asn1:"optional,default:1"`
}
// from PKCS#7:
type digestInfo struct {
Algorithm pkix.AlgorithmIdentifier
Digest []byte
}
var (
oidSHA1 = asn1.ObjectIdentifier([]int{1, 3, 14, 3, 2, 26})
)
func verifyMac(macData *macData, message, password []byte) error {
if !macData.Mac.Algorithm.Algorithm.Equal(oidSHA1) {
return NotImplementedError("unknown digest algorithm: " + macData.Mac.Algorithm.Algorithm.String())
}
key := pbkdf(sha1Sum, 20, 64, macData.MacSalt, password, macData.Iterations, 3, 20)
mac := hmac.New(sha1.New, key)
mac.Write(message)
expectedMAC := mac.Sum(nil)
if !hmac.Equal(macData.Mac.Digest, expectedMAC) {
return ErrIncorrectPassword
}
return nil
}
func computeMac(macData *macData, message, password []byte) error {
if !macData.Mac.Algorithm.Algorithm.Equal(oidSHA1) {
return NotImplementedError("unknown digest algorithm: " + macData.Mac.Algorithm.Algorithm.String())
}
key := pbkdf(sha1Sum, 20, 64, macData.MacSalt, password, macData.Iterations, 3, 20)
mac := hmac.New(sha1.New, key)
mac.Write(message)
macData.Mac.Digest = mac.Sum(nil)
return nil
}

73
vendor/software.sslmate.com/src/go-pkcs12/mac_test.go generated vendored Normal file
View File

@@ -0,0 +1,73 @@
// Copyright 2015, 2018, 2019 Opsmate, Inc. All rights reserved.
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package pkcs12
import (
"bytes"
"encoding/asn1"
"testing"
)
func TestVerifyMac(t *testing.T) {
td := macData{
Mac: digestInfo{
Digest: []byte{0x18, 0x20, 0x3d, 0xff, 0x1e, 0x16, 0xf4, 0x92, 0xf2, 0xaf, 0xc8, 0x91, 0xa9, 0xba, 0xd6, 0xca, 0x9d, 0xee, 0x51, 0x93},
},
MacSalt: []byte{1, 2, 3, 4, 5, 6, 7, 8},
Iterations: 2048,
}
message := []byte{11, 12, 13, 14, 15}
password, _ := bmpString("")
td.Mac.Algorithm.Algorithm = asn1.ObjectIdentifier([]int{1, 2, 3})
err := verifyMac(&td, message, password)
if _, ok := err.(NotImplementedError); !ok {
t.Errorf("err: %v", err)
}
td.Mac.Algorithm.Algorithm = asn1.ObjectIdentifier([]int{1, 3, 14, 3, 2, 26})
err = verifyMac(&td, message, password)
if err != ErrIncorrectPassword {
t.Errorf("Expected incorrect password, got err: %v", err)
}
password, _ = bmpString("Sesame open")
err = verifyMac(&td, message, password)
if err != nil {
t.Errorf("err: %v", err)
}
}
func TestComputeMac(t *testing.T) {
td := macData{
MacSalt: []byte{1, 2, 3, 4, 5, 6, 7, 8},
Iterations: 2048,
}
message := []byte{11, 12, 13, 14, 15}
password, _ := bmpString("Sesame open")
td.Mac.Algorithm.Algorithm = asn1.ObjectIdentifier([]int{1, 2, 3})
err := computeMac(&td, message, password)
if _, ok := err.(NotImplementedError); !ok {
t.Errorf("err: %v", err)
}
td.Mac.Algorithm.Algorithm = asn1.ObjectIdentifier([]int{1, 3, 14, 3, 2, 26})
err = computeMac(&td, message, password)
if err != nil {
t.Errorf("err: %v", err)
}
expectedDigest := []byte{0x18, 0x20, 0x3d, 0xff, 0x1e, 0x16, 0xf4, 0x92, 0xf2, 0xaf, 0xc8, 0x91, 0xa9, 0xba, 0xd6, 0xca, 0x9d, 0xee, 0x51, 0x93}
if bytes.Compare(td.Mac.Digest, expectedDigest) != 0 {
t.Errorf("Computed incorrect MAC; expected MAC to be '%d' but got '%d'", expectedDigest, td.Mac.Digest)
}
}

170
vendor/software.sslmate.com/src/go-pkcs12/pbkdf.go generated vendored Normal file
View File

@@ -0,0 +1,170 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package pkcs12
import (
"bytes"
"crypto/sha1"
"math/big"
)
var (
one = big.NewInt(1)
)
// sha1Sum returns the SHA-1 hash of in.
func sha1Sum(in []byte) []byte {
sum := sha1.Sum(in)
return sum[:]
}
// fillWithRepeats returns v*ceiling(len(pattern) / v) bytes consisting of
// repeats of pattern.
func fillWithRepeats(pattern []byte, v int) []byte {
if len(pattern) == 0 {
return nil
}
outputLen := v * ((len(pattern) + v - 1) / v)
return bytes.Repeat(pattern, (outputLen+len(pattern)-1)/len(pattern))[:outputLen]
}
func pbkdf(hash func([]byte) []byte, u, v int, salt, password []byte, r int, ID byte, size int) (key []byte) {
// implementation of https://tools.ietf.org/html/rfc7292#appendix-B.2 , RFC text verbatim in comments
// Let H be a hash function built around a compression function f:
// Z_2^u x Z_2^v -> Z_2^u
// (that is, H has a chaining variable and output of length u bits, and
// the message input to the compression function of H is v bits). The
// values for u and v are as follows:
// HASH FUNCTION VALUE u VALUE v
// MD2, MD5 128 512
// SHA-1 160 512
// SHA-224 224 512
// SHA-256 256 512
// SHA-384 384 1024
// SHA-512 512 1024
// SHA-512/224 224 1024
// SHA-512/256 256 1024
// Furthermore, let r be the iteration count.
// We assume here that u and v are both multiples of 8, as are the
// lengths of the password and salt strings (which we denote by p and s,
// respectively) and the number n of pseudorandom bits required. In
// addition, u and v are of course non-zero.
// For information on security considerations for MD5 [19], see [25] and
// [1], and on those for MD2, see [18].
// The following procedure can be used to produce pseudorandom bits for
// a particular "purpose" that is identified by a byte called "ID".
// This standard specifies 3 different values for the ID byte:
// 1. If ID=1, then the pseudorandom bits being produced are to be used
// as key material for performing encryption or decryption.
// 2. If ID=2, then the pseudorandom bits being produced are to be used
// as an IV (Initial Value) for encryption or decryption.
// 3. If ID=3, then the pseudorandom bits being produced are to be used
// as an integrity key for MACing.
// 1. Construct a string, D (the "diversifier"), by concatenating v/8
// copies of ID.
var D []byte
for i := 0; i < v; i++ {
D = append(D, ID)
}
// 2. Concatenate copies of the salt together to create a string S of
// length v(ceiling(s/v)) bits (the final copy of the salt may be
// truncated to create S). Note that if the salt is the empty
// string, then so is S.
S := fillWithRepeats(salt, v)
// 3. Concatenate copies of the password together to create a string P
// of length v(ceiling(p/v)) bits (the final copy of the password
// may be truncated to create P). Note that if the password is the
// empty string, then so is P.
P := fillWithRepeats(password, v)
// 4. Set I=S||P to be the concatenation of S and P.
I := append(S, P...)
// 5. Set c=ceiling(n/u).
c := (size + u - 1) / u
// 6. For i=1, 2, ..., c, do the following:
A := make([]byte, c*20)
var IjBuf []byte
for i := 0; i < c; i++ {
// A. Set A2=H^r(D||I). (i.e., the r-th hash of D||1,
// H(H(H(... H(D||I))))
Ai := hash(append(D, I...))
for j := 1; j < r; j++ {
Ai = hash(Ai)
}
copy(A[i*20:], Ai[:])
if i < c-1 { // skip on last iteration
// B. Concatenate copies of Ai to create a string B of length v
// bits (the final copy of Ai may be truncated to create B).
var B []byte
for len(B) < v {
B = append(B, Ai[:]...)
}
B = B[:v]
// C. Treating I as a concatenation I_0, I_1, ..., I_(k-1) of v-bit
// blocks, where k=ceiling(s/v)+ceiling(p/v), modify I by
// setting I_j=(I_j+B+1) mod 2^v for each j.
{
Bbi := new(big.Int).SetBytes(B)
Ij := new(big.Int)
for j := 0; j < len(I)/v; j++ {
Ij.SetBytes(I[j*v : (j+1)*v])
Ij.Add(Ij, Bbi)
Ij.Add(Ij, one)
Ijb := Ij.Bytes()
// We expect Ijb to be exactly v bytes,
// if it is longer or shorter we must
// adjust it accordingly.
if len(Ijb) > v {
Ijb = Ijb[len(Ijb)-v:]
}
if len(Ijb) < v {
if IjBuf == nil {
IjBuf = make([]byte, v)
}
bytesShort := v - len(Ijb)
for i := 0; i < bytesShort; i++ {
IjBuf[i] = 0
}
copy(IjBuf[bytesShort:], Ijb)
Ijb = IjBuf
}
copy(I[j*v:(j+1)*v], Ijb)
}
}
}
}
// 7. Concatenate A_1, A_2, ..., A_c together to form a pseudorandom
// bit string, A.
// 8. Use the first n bits of A as the output of this entire process.
return A[:size]
// If the above process is being used to generate a DES key, the process
// should be used to create 64 random bits, and the key's parity bits
// should be set after the 64 bits have been produced. Similar concerns
// hold for 2-key and 3-key triple-DES keys, for CDMF keys, and for any
// similar keys with parity bits "built into them".
}

View File

@@ -0,0 +1,34 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package pkcs12
import (
"bytes"
"testing"
)
func TestThatPBKDFWorksCorrectlyForLongKeys(t *testing.T) {
cipherInfo := shaWithTripleDESCBC{}
salt := []byte("\xff\xff\xff\xff\xff\xff\xff\xff")
password, _ := bmpString("sesame")
key := cipherInfo.deriveKey(salt, password, 2048)
if expected := []byte("\x7c\xd9\xfd\x3e\x2b\x3b\xe7\x69\x1a\x44\xe3\xbe\xf0\xf9\xea\x0f\xb9\xb8\x97\xd4\xe3\x25\xd9\xd1"); bytes.Compare(key, expected) != 0 {
t.Fatalf("expected key '%x', but found '%x'", expected, key)
}
}
func TestThatPBKDFHandlesLeadingZeros(t *testing.T) {
// This test triggers a case where I_j (in step 6C) ends up with leading zero
// byte, meaning that len(Ijb) < v (leading zeros get stripped by big.Int).
// This was previously causing bug whereby certain inputs would break the
// derivation and produce the wrong output.
key := pbkdf(sha1Sum, 20, 64, []byte("\xf3\x7e\x05\xb5\x18\x32\x4b\x4b"), []byte("\x00\x00"), 2048, 1, 24)
expected := []byte("\x00\xf7\x59\xff\x47\xd1\x4d\xd0\x36\x65\xd5\x94\x3c\xb3\xc4\xa3\x9a\x25\x55\xc0\x2a\xed\x66\xe1")
if bytes.Compare(key, expected) != 0 {
t.Fatalf("expected key '%x', but found '%x'", expected, key)
}
}

540
vendor/software.sslmate.com/src/go-pkcs12/pkcs12.go generated vendored Normal file
View File

@@ -0,0 +1,540 @@
// Copyright 2015, 2018, 2019 Opsmate, Inc. All rights reserved.
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package pkcs12 implements some of PKCS#12 (also known as P12 or PFX).
// It is intended for decoding P12/PFX files for use with the crypto/tls
// package, and for encoding P12/PFX files for use by legacy applications which
// do not support newer formats. Since PKCS#12 uses weak encryption
// primitives, it SHOULD NOT be used for new applications.
//
// This package is forked from golang.org/x/crypto/pkcs12, which is frozen.
// The implementation is distilled from https://tools.ietf.org/html/rfc7292
// and referenced documents.
package pkcs12 // import "software.sslmate.com/src/go-pkcs12"
import (
"crypto/ecdsa"
"crypto/rsa"
"crypto/sha1"
"crypto/x509"
"crypto/x509/pkix"
"encoding/asn1"
"encoding/hex"
"encoding/pem"
"errors"
"io"
)
// DefaultPassword is the string "changeit", a commonly-used password for
// PKCS#12 files. Due to the weak encryption used by PKCS#12, it is
// RECOMMENDED that you use DefaultPassword when encoding PKCS#12 files,
// and protect the PKCS#12 files using other means.
const DefaultPassword = "changeit"
var (
oidDataContentType = asn1.ObjectIdentifier([]int{1, 2, 840, 113549, 1, 7, 1})
oidEncryptedDataContentType = asn1.ObjectIdentifier([]int{1, 2, 840, 113549, 1, 7, 6})
oidFriendlyName = asn1.ObjectIdentifier([]int{1, 2, 840, 113549, 1, 9, 20})
oidLocalKeyID = asn1.ObjectIdentifier([]int{1, 2, 840, 113549, 1, 9, 21})
oidMicrosoftCSPName = asn1.ObjectIdentifier([]int{1, 3, 6, 1, 4, 1, 311, 17, 1})
)
type pfxPdu struct {
Version int
AuthSafe contentInfo
MacData macData `asn1:"optional"`
}
type contentInfo struct {
ContentType asn1.ObjectIdentifier
Content asn1.RawValue `asn1:"tag:0,explicit,optional"`
}
type encryptedData struct {
Version int
EncryptedContentInfo encryptedContentInfo
}
type encryptedContentInfo struct {
ContentType asn1.ObjectIdentifier
ContentEncryptionAlgorithm pkix.AlgorithmIdentifier
EncryptedContent []byte `asn1:"tag:0,optional"`
}
func (i encryptedContentInfo) Algorithm() pkix.AlgorithmIdentifier {
return i.ContentEncryptionAlgorithm
}
func (i encryptedContentInfo) Data() []byte { return i.EncryptedContent }
func (i *encryptedContentInfo) SetData(data []byte) { i.EncryptedContent = data }
type safeBag struct {
Id asn1.ObjectIdentifier
Value asn1.RawValue `asn1:"tag:0,explicit"`
Attributes []pkcs12Attribute `asn1:"set,optional"`
}
type pkcs12Attribute struct {
Id asn1.ObjectIdentifier
Value asn1.RawValue `asn1:"set"`
}
type encryptedPrivateKeyInfo struct {
AlgorithmIdentifier pkix.AlgorithmIdentifier
EncryptedData []byte
}
func (i encryptedPrivateKeyInfo) Algorithm() pkix.AlgorithmIdentifier {
return i.AlgorithmIdentifier
}
func (i encryptedPrivateKeyInfo) Data() []byte {
return i.EncryptedData
}
func (i *encryptedPrivateKeyInfo) SetData(data []byte) {
i.EncryptedData = data
}
// PEM block types
const (
certificateType = "CERTIFICATE"
privateKeyType = "PRIVATE KEY"
)
// unmarshal calls asn1.Unmarshal, but also returns an error if there is any
// trailing data after unmarshaling.
func unmarshal(in []byte, out interface{}) error {
trailing, err := asn1.Unmarshal(in, out)
if err != nil {
return err
}
if len(trailing) != 0 {
return errors.New("pkcs12: trailing data found")
}
return nil
}
// ToPEM converts all "safe bags" contained in pfxData to PEM blocks.
// DO NOT USE THIS FUNCTION. ToPEM creates invalid PEM blocks; private keys
// are encoded as raw RSA or EC private keys rather than PKCS#8 despite being
// labeled "PRIVATE KEY". To decode a PKCS#12 file, use DecodeChain instead,
// and use the encoding/pem package to convert to PEM if necessary.
func ToPEM(pfxData []byte, password string) ([]*pem.Block, error) {
encodedPassword, err := bmpString(password)
if err != nil {
return nil, ErrIncorrectPassword
}
bags, encodedPassword, err := getSafeContents(pfxData, encodedPassword)
if err != nil {
return nil, err
}
blocks := make([]*pem.Block, 0, len(bags))
for _, bag := range bags {
block, err := convertBag(&bag, encodedPassword)
if err != nil {
return nil, err
}
blocks = append(blocks, block)
}
return blocks, nil
}
func convertBag(bag *safeBag, password []byte) (*pem.Block, error) {
block := &pem.Block{
Headers: make(map[string]string),
}
for _, attribute := range bag.Attributes {
k, v, err := convertAttribute(&attribute)
if err != nil {
return nil, err
}
block.Headers[k] = v
}
switch {
case bag.Id.Equal(oidCertBag):
block.Type = certificateType
certsData, err := decodeCertBag(bag.Value.Bytes)
if err != nil {
return nil, err
}
block.Bytes = certsData
case bag.Id.Equal(oidPKCS8ShroundedKeyBag):
block.Type = privateKeyType
key, err := decodePkcs8ShroudedKeyBag(bag.Value.Bytes, password)
if err != nil {
return nil, err
}
switch key := key.(type) {
case *rsa.PrivateKey:
block.Bytes = x509.MarshalPKCS1PrivateKey(key)
case *ecdsa.PrivateKey:
block.Bytes, err = x509.MarshalECPrivateKey(key)
if err != nil {
return nil, err
}
default:
return nil, errors.New("found unknown private key type in PKCS#8 wrapping")
}
default:
return nil, errors.New("don't know how to convert a safe bag of type " + bag.Id.String())
}
return block, nil
}
func convertAttribute(attribute *pkcs12Attribute) (key, value string, err error) {
isString := false
switch {
case attribute.Id.Equal(oidFriendlyName):
key = "friendlyName"
isString = true
case attribute.Id.Equal(oidLocalKeyID):
key = "localKeyId"
case attribute.Id.Equal(oidMicrosoftCSPName):
// This key is chosen to match OpenSSL.
key = "Microsoft CSP Name"
isString = true
default:
return "", "", errors.New("pkcs12: unknown attribute with OID " + attribute.Id.String())
}
if isString {
if err := unmarshal(attribute.Value.Bytes, &attribute.Value); err != nil {
return "", "", err
}
if value, err = decodeBMPString(attribute.Value.Bytes); err != nil {
return "", "", err
}
} else {
var id []byte
if err := unmarshal(attribute.Value.Bytes, &id); err != nil {
return "", "", err
}
value = hex.EncodeToString(id)
}
return key, value, nil
}
// Decode extracts a certificate and private key from pfxData. This function
// assumes that there is only one certificate and only one private key in the
// pfxData. Since PKCS#12 files often contain more than one certificate, you
// probably want to use DecodeChain instead.
func Decode(pfxData []byte, password string) (privateKey interface{}, certificate *x509.Certificate, err error) {
var caCerts []*x509.Certificate
privateKey, certificate, caCerts, err = DecodeChain(pfxData, password)
if len(caCerts) != 0 {
err = errors.New("pkcs12: expected exactly two safe bags in the PFX PDU")
}
return
}
// DecodeChain extracts a certificate, a CA certificate chain, and private key
// from pfxData. This function assumes that there is at least one certificate
// and only one private key in the pfxData. The first certificate is assumed to
// be the leaf certificate, and subsequent certificates, if any, are assumed to
// comprise the CA certificate chain.
func DecodeChain(pfxData []byte, password string) (privateKey interface{}, certificate *x509.Certificate, caCerts []*x509.Certificate, err error) {
encodedPassword, err := bmpString(password)
if err != nil {
return nil, nil, nil, err
}
bags, encodedPassword, err := getSafeContents(pfxData, encodedPassword)
if err != nil {
return nil, nil, nil, err
}
for _, bag := range bags {
switch {
case bag.Id.Equal(oidCertBag):
certsData, err := decodeCertBag(bag.Value.Bytes)
if err != nil {
return nil, nil, nil, err
}
certs, err := x509.ParseCertificates(certsData)
if err != nil {
return nil, nil, nil, err
}
if len(certs) != 1 {
err = errors.New("pkcs12: expected exactly one certificate in the certBag")
return nil, nil, nil, err
}
if certificate == nil {
certificate = certs[0]
} else {
caCerts = append(caCerts, certs[0])
}
case bag.Id.Equal(oidPKCS8ShroundedKeyBag):
if privateKey != nil {
err = errors.New("pkcs12: expected exactly one key bag")
return nil, nil, nil, err
}
if privateKey, err = decodePkcs8ShroudedKeyBag(bag.Value.Bytes, encodedPassword); err != nil {
return nil, nil, nil, err
}
}
}
if certificate == nil {
return nil, nil, nil, errors.New("pkcs12: certificate missing")
}
if privateKey == nil {
return nil, nil, nil, errors.New("pkcs12: private key missing")
}
return
}
func getSafeContents(p12Data, password []byte) (bags []safeBag, updatedPassword []byte, err error) {
pfx := new(pfxPdu)
if err := unmarshal(p12Data, pfx); err != nil {
return nil, nil, errors.New("pkcs12: error reading P12 data: " + err.Error())
}
if pfx.Version != 3 {
return nil, nil, NotImplementedError("can only decode v3 PFX PDU's")
}
if !pfx.AuthSafe.ContentType.Equal(oidDataContentType) {
return nil, nil, NotImplementedError("only password-protected PFX is implemented")
}
// unmarshal the explicit bytes in the content for type 'data'
if err := unmarshal(pfx.AuthSafe.Content.Bytes, &pfx.AuthSafe.Content); err != nil {
return nil, nil, err
}
if len(pfx.MacData.Mac.Algorithm.Algorithm) == 0 {
return nil, nil, errors.New("pkcs12: no MAC in data")
}
if err := verifyMac(&pfx.MacData, pfx.AuthSafe.Content.Bytes, password); err != nil {
if err == ErrIncorrectPassword && len(password) == 2 && password[0] == 0 && password[1] == 0 {
// some implementations use an empty byte array
// for the empty string password try one more
// time with empty-empty password
password = nil
err = verifyMac(&pfx.MacData, pfx.AuthSafe.Content.Bytes, password)
}
if err != nil {
return nil, nil, err
}
}
var authenticatedSafe []contentInfo
if err := unmarshal(pfx.AuthSafe.Content.Bytes, &authenticatedSafe); err != nil {
return nil, nil, err
}
if len(authenticatedSafe) != 2 {
return nil, nil, NotImplementedError("expected exactly two items in the authenticated safe")
}
for _, ci := range authenticatedSafe {
var data []byte
switch {
case ci.ContentType.Equal(oidDataContentType):
if err := unmarshal(ci.Content.Bytes, &data); err != nil {
return nil, nil, err
}
case ci.ContentType.Equal(oidEncryptedDataContentType):
var encryptedData encryptedData
if err := unmarshal(ci.Content.Bytes, &encryptedData); err != nil {
return nil, nil, err
}
if encryptedData.Version != 0 {
return nil, nil, NotImplementedError("only version 0 of EncryptedData is supported")
}
if data, err = pbDecrypt(encryptedData.EncryptedContentInfo, password); err != nil {
return nil, nil, err
}
default:
return nil, nil, NotImplementedError("only data and encryptedData content types are supported in authenticated safe")
}
var safeContents []safeBag
if err := unmarshal(data, &safeContents); err != nil {
return nil, nil, err
}
bags = append(bags, safeContents...)
}
return bags, password, nil
}
// Encode produces pfxData containing one private key (privateKey), an
// end-entity certificate (certificate), and any number of CA certificates
// (caCerts).
//
// The private key is encrypted with the provided password, but due to the
// weak encryption primitives used by PKCS#12, it is RECOMMENDED that you
// specify a hard-coded password (such as pkcs12.DefaultPassword) and protect
// the resulting pfxData using other means.
//
// The rand argument is used to provide entropy for the encryption, and
// can be set to rand.Reader from the crypto/rand package.
//
// Encode emulates the behavior of OpenSSL's PKCS12_create: it creates two
// SafeContents: one that's encrypted with RC2 and contains the certificates,
// and another that is unencrypted and contains the private key shrouded with
// 3DES The private key bag and the end-entity certificate bag have the
// LocalKeyId attribute set to the SHA-1 fingerprint of the end-entity
// certificate.
func Encode(rand io.Reader, privateKey interface{}, certificate *x509.Certificate, caCerts []*x509.Certificate, password string) (pfxData []byte, err error) {
encodedPassword, err := bmpString(password)
if err != nil {
return nil, err
}
var pfx pfxPdu
pfx.Version = 3
var certFingerprint = sha1.Sum(certificate.Raw)
var localKeyIdAttr pkcs12Attribute
localKeyIdAttr.Id = oidLocalKeyID
localKeyIdAttr.Value.Class = 0
localKeyIdAttr.Value.Tag = 17
localKeyIdAttr.Value.IsCompound = true
if localKeyIdAttr.Value.Bytes, err = asn1.Marshal(certFingerprint[:]); err != nil {
return nil, err
}
var certBags []safeBag
var certBag *safeBag
if certBag, err = makeCertBag(certificate.Raw, []pkcs12Attribute{localKeyIdAttr}); err != nil {
return nil, err
}
certBags = append(certBags, *certBag)
for _, cert := range caCerts {
if certBag, err = makeCertBag(cert.Raw, []pkcs12Attribute{}); err != nil {
return nil, err
}
certBags = append(certBags, *certBag)
}
var keyBag safeBag
keyBag.Id = oidPKCS8ShroundedKeyBag
keyBag.Value.Class = 2
keyBag.Value.Tag = 0
keyBag.Value.IsCompound = true
if keyBag.Value.Bytes, err = encodePkcs8ShroudedKeyBag(rand, privateKey, encodedPassword); err != nil {
return nil, err
}
keyBag.Attributes = append(keyBag.Attributes, localKeyIdAttr)
// Construct an authenticated safe with two SafeContents.
// The first SafeContents is encrypted and contains the cert bags.
// The second SafeContents is unencrypted and contains the shrouded key bag.
var authenticatedSafe [2]contentInfo
if authenticatedSafe[0], err = makeSafeContents(rand, certBags, encodedPassword); err != nil {
return nil, err
}
if authenticatedSafe[1], err = makeSafeContents(rand, []safeBag{keyBag}, nil); err != nil {
return nil, err
}
var authenticatedSafeBytes []byte
if authenticatedSafeBytes, err = asn1.Marshal(authenticatedSafe[:]); err != nil {
return nil, err
}
// compute the MAC
pfx.MacData.Mac.Algorithm.Algorithm = oidSHA1
pfx.MacData.MacSalt = make([]byte, 8)
if _, err = rand.Read(pfx.MacData.MacSalt); err != nil {
return nil, err
}
pfx.MacData.Iterations = 1
if err = computeMac(&pfx.MacData, authenticatedSafeBytes, encodedPassword); err != nil {
return nil, err
}
pfx.AuthSafe.ContentType = oidDataContentType
pfx.AuthSafe.Content.Class = 2
pfx.AuthSafe.Content.Tag = 0
pfx.AuthSafe.Content.IsCompound = true
if pfx.AuthSafe.Content.Bytes, err = asn1.Marshal(authenticatedSafeBytes); err != nil {
return nil, err
}
if pfxData, err = asn1.Marshal(pfx); err != nil {
return nil, errors.New("pkcs12: error writing P12 data: " + err.Error())
}
return
}
func makeCertBag(certBytes []byte, attributes []pkcs12Attribute) (certBag *safeBag, err error) {
certBag = new(safeBag)
certBag.Id = oidCertBag
certBag.Value.Class = 2
certBag.Value.Tag = 0
certBag.Value.IsCompound = true
if certBag.Value.Bytes, err = encodeCertBag(certBytes); err != nil {
return nil, err
}
certBag.Attributes = attributes
return
}
func makeSafeContents(rand io.Reader, bags []safeBag, password []byte) (ci contentInfo, err error) {
var data []byte
if data, err = asn1.Marshal(bags); err != nil {
return
}
if password == nil {
ci.ContentType = oidDataContentType
ci.Content.Class = 2
ci.Content.Tag = 0
ci.Content.IsCompound = true
if ci.Content.Bytes, err = asn1.Marshal(data); err != nil {
return
}
} else {
randomSalt := make([]byte, 8)
if _, err = rand.Read(randomSalt); err != nil {
return
}
var algo pkix.AlgorithmIdentifier
algo.Algorithm = oidPBEWithSHAAnd40BitRC2CBC
if algo.Parameters.FullBytes, err = asn1.Marshal(pbeParams{Salt: randomSalt, Iterations: 2048}); err != nil {
return
}
var encryptedData encryptedData
encryptedData.Version = 0
encryptedData.EncryptedContentInfo.ContentType = oidDataContentType
encryptedData.EncryptedContentInfo.ContentEncryptionAlgorithm = algo
if err = pbEncrypt(&encryptedData.EncryptedContentInfo, data, password); err != nil {
return
}
ci.ContentType = oidEncryptedDataContentType
ci.Content.Class = 2
ci.Content.Tag = 0
ci.Content.IsCompound = true
if ci.Content.Bytes, err = asn1.Marshal(encryptedData); err != nil {
return
}
}
return
}

View File

@@ -0,0 +1,138 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package pkcs12
import (
"crypto/rsa"
"crypto/tls"
"encoding/base64"
"encoding/pem"
"testing"
)
func TestPfx(t *testing.T) {
for commonName, base64P12 := range testdata {
p12, _ := base64.StdEncoding.DecodeString(base64P12)
priv, cert, err := Decode(p12, "")
if err != nil {
t.Fatal(err)
}
if err := priv.(*rsa.PrivateKey).Validate(); err != nil {
t.Errorf("error while validating private key: %v", err)
}
if cert.Subject.CommonName != commonName {
t.Errorf("expected common name to be %q, but found %q", commonName, cert.Subject.CommonName)
}
}
}
func TestPEM(t *testing.T) {
for commonName, base64P12 := range testdata {
p12, _ := base64.StdEncoding.DecodeString(base64P12)
blocks, err := ToPEM(p12, "")
if err != nil {
t.Fatalf("error while converting to PEM: %s", err)
}
var pemData []byte
for _, b := range blocks {
pemData = append(pemData, pem.EncodeToMemory(b)...)
}
cert, err := tls.X509KeyPair(pemData, pemData)
if err != nil {
t.Errorf("err while converting to key pair: %v", err)
}
config := tls.Config{
Certificates: []tls.Certificate{cert},
}
config.BuildNameToCertificate()
if _, exists := config.NameToCertificate[commonName]; !exists {
t.Errorf("did not find our cert in PEM?: %v", config.NameToCertificate)
}
}
}
func ExampleToPEM() {
p12, _ := base64.StdEncoding.DecodeString(`MIIJzgIBAzCCCZQGCS ... CA+gwggPk==`)
blocks, err := ToPEM(p12, "password")
if err != nil {
panic(err)
}
var pemData []byte
for _, b := range blocks {
pemData = append(pemData, pem.EncodeToMemory(b)...)
}
// then use PEM data for tls to construct tls certificate:
cert, err := tls.X509KeyPair(pemData, pemData)
if err != nil {
panic(err)
}
config := &tls.Config{
Certificates: []tls.Certificate{cert},
}
_ = config
}
var testdata = map[string]string{
// 'null' password test case
"Windows Azure Tools": `MIIKDAIBAzCCCcwGCSqGSIb3DQEHAaCCCb0Eggm5MIIJtTCCBe4GCSqGSIb3DQEHAaCCBd8EggXbMIIF1zCCBdMGCyqGSIb3DQEMCgECoIIE7jCCBOowHAYKKoZIhvcNAQwBAzAOBAhStUNnlTGV+gICB9AEggTIJ81JIossF6boFWpPtkiQRPtI6DW6e9QD4/WvHAVrM2bKdpMzSMsCML5NyuddANTKHBVq00Jc9keqGNAqJPKkjhSUebzQFyhe0E1oI9T4zY5UKr/I8JclOeccH4QQnsySzYUG2SnniXnQ+JrG3juetli7EKth9h6jLc6xbubPadY5HMB3wL/eG/kJymiXwU2KQ9Mgd4X6jbcV+NNCE/8jbZHvSTCPeYTJIjxfeX61Sj5kFKUCzERbsnpyevhY3X0eYtEDezZQarvGmXtMMdzf8HJHkWRdk9VLDLgjk8uiJif/+X4FohZ37ig0CpgC2+dP4DGugaZZ51hb8tN9GeCKIsrmWogMXDIVd0OACBp/EjJVmFB6y0kUCXxUE0TZt0XA1tjAGJcjDUpBvTntZjPsnH/4ZySy+s2d9OOhJ6pzRQBRm360TzkFdSwk9DLiLdGfv4pwMMu/vNGBlqjP/1sQtj+jprJiD1sDbCl4AdQZVoMBQHadF2uSD4/o17XG/Ci0r2h6Htc2yvZMAbEY4zMjjIn2a+vqIxD6onexaek1R3zbkS9j19D6EN9EWn8xgz80YRCyW65znZk8xaIhhvlU/mg7sTxeyuqroBZNcq6uDaQTehDpyH7bY2l4zWRpoj10a6JfH2q5shYz8Y6UZC/kOTfuGqbZDNZWro/9pYquvNNW0M847E5t9bsf9VkAAMHRGBbWoVoU9VpI0UnoXSfvpOo+aXa2DSq5sHHUTVY7A9eov3z5IqT+pligx11xcs+YhDWcU8di3BTJisohKvv5Y8WSkm/rloiZd4ig269k0jTRk1olP/vCksPli4wKG2wdsd5o42nX1yL7mFfXocOANZbB+5qMkiwdyoQSk+Vq+C8nAZx2bbKhUq2MbrORGMzOe0Hh0x2a0PeObycN1Bpyv7Mp3ZI9h5hBnONKCnqMhtyQHUj/nNvbJUnDVYNfoOEqDiEqqEwB7YqWzAKz8KW0OIqdlM8uiQ4JqZZlFllnWJUfaiDrdFM3lYSnFQBkzeVlts6GpDOOBjCYd7dcCNS6kq6pZC6p6HN60Twu0JnurZD6RT7rrPkIGE8vAenFt4iGe/yF52fahCSY8Ws4K0UTwN7bAS+4xRHVCWvE8sMRZsRCHizb5laYsVrPZJhE6+hux6OBb6w8kwPYXc+ud5v6UxawUWgt6uPwl8mlAtU9Z7Miw4Nn/wtBkiLL/ke1UI1gqJtcQXgHxx6mzsjh41+nAgTvdbsSEyU6vfOmxGj3Rwc1eOrIhJUqn5YjOWfzzsz/D5DzWKmwXIwdspt1p+u+kol1N3f2wT9fKPnd/RGCb4g/1hc3Aju4DQYgGY782l89CEEdalpQ/35bQczMFk6Fje12HykakWEXd/bGm9Unh82gH84USiRpeOfQvBDYoqEyrY3zkFZzBjhDqa+jEcAj41tcGx47oSfDq3iVYCdL7HSIjtnyEktVXd7mISZLoMt20JACFcMw+mrbjlug+eU7o2GR7T+LwtOp/p4LZqyLa7oQJDwde1BNZtm3TCK2P1mW94QDL0nDUps5KLtr1DaZXEkRbjSJub2ZE9WqDHyU3KA8G84Tq/rN1IoNu/if45jacyPje1Npj9IftUZSP22nV7HMwZtwQ4P4MYHRMBMGCSqGSIb3DQEJFTEGBAQBAAAAMFsGCSqGSIb3DQEJFDFOHkwAewBCADQAQQA0AEYARQBCADAALQBBADEAOABBAC0ANAA0AEIAQgAtAEIANQBGADIALQA0ADkAMQBFAEYAMQA1ADIAQgBBADEANgB9MF0GCSsGAQQBgjcRATFQHk4ATQBpAGMAcgBvAHMAbwBmAHQAIABTAG8AZgB0AHcAYQByAGUAIABLAGUAeQAgAFMAdABvAHIAYQBnAGUAIABQAHIAbwB2AGkAZABlAHIwggO/BgkqhkiG9w0BBwagggOwMIIDrAIBADCCA6UGCSqGSIb3DQEHATAcBgoqhkiG9w0BDAEGMA4ECEBk5ZAYpu0WAgIH0ICCA3hik4mQFGpw9Ha8TQPtk+j2jwWdxfF0+sTk6S8PTsEfIhB7wPltjiCK92Uv2tCBQnodBUmatIfkpnRDEySmgmdglmOCzj204lWAMRs94PoALGn3JVBXbO1vIDCbAPOZ7Z0Hd0/1t2hmk8v3//QJGUg+qr59/4y/MuVfIg4qfkPcC2QSvYWcK3oTf6SFi5rv9B1IOWFgN5D0+C+x/9Lb/myPYX+rbOHrwtJ4W1fWKoz9g7wwmGFA9IJ2DYGuH8ifVFbDFT1Vcgsvs8arSX7oBsJVW0qrP7XkuDRe3EqCmKW7rBEwYrFznhxZcRDEpMwbFoSvgSIZ4XhFY9VKYglT+JpNH5iDceYEBOQL4vBLpxNUk3l5jKaBNxVa14AIBxq18bVHJ+STInhLhad4u10v/Xbx7wIL3f9DX1yLAkPrpBYbNHS2/ew6H/ySDJnoIDxkw2zZ4qJ+qUJZ1S0lbZVG+VT0OP5uF6tyOSpbMlcGkdl3z254n6MlCrTifcwkzscysDsgKXaYQw06rzrPW6RDub+t+hXzGny799fS9jhQMLDmOggaQ7+LA4oEZsfT89HLMWxJYDqjo3gIfjciV2mV54R684qLDS+AO09U49e6yEbwGlq8lpmO/pbXCbpGbB1b3EomcQbxdWxW2WEkkEd/VBn81K4M3obmywwXJkw+tPXDXfBmzzaqqCR+onMQ5ME1nMkY8ybnfoCc1bDIupjVWsEL2Wvq752RgI6KqzVNr1ew1IdqV5AWN2fOfek+0vi3Jd9FHF3hx8JMwjJL9dZsETV5kHtYJtE7wJ23J68BnCt2eI0GEuwXcCf5EdSKN/xXCTlIokc4Qk/gzRdIZsvcEJ6B1lGovKG54X4IohikqTjiepjbsMWj38yxDmK3mtENZ9ci8FPfbbvIEcOCZIinuY3qFUlRSbx7VUerEoV1IP3clUwexVQo4lHFee2jd7ocWsdSqSapW7OWUupBtDzRkqVhE7tGria+i1W2d6YLlJ21QTjyapWJehAMO637OdbJCCzDs1cXbodRRE7bsP492ocJy8OX66rKdhYbg8srSFNKdb3pF3UDNbN9jhI/t8iagRhNBhlQtTr1me2E/c86Q18qcRXl4bcXTt6acgCeffK6Y26LcVlrgjlD33AEYRRUeyC+rpxbT0aMjdFderlndKRIyG23mSp0HaUwNzAfMAcGBSsOAwIaBBRlviCbIyRrhIysg2dc/KbLFTc2vQQUg4rfwHMM4IKYRD/fsd1x6dda+wQ=`,
// empty string password test case
"testing@example.com": `MIIJzgIBAzCCCZQGCSqGSIb3DQEHAaCCCYUEggmBMIIJfTCCA/cGCSqGSIb3DQEHBqCCA+gwggPk
AgEAMIID3QYJKoZIhvcNAQcBMBwGCiqGSIb3DQEMAQYwDgQIIszfRGqcmPcCAggAgIIDsOZ9Eg1L
s5Wx8JhYoV3HAL4aRnkAWvTYB5NISZOgSgIQTssmt/3A7134dibTmaT/93LikkL3cTKLnQzJ4wDf
YZ1bprpVJvUqz+HFT79m27bP9zYXFrvxWBJbxjYKTSjQMgz+h8LAEpXXGajCmxMJ1oCOtdXkhhzc
LdZN6SAYgtmtyFnCdMEDskSggGuLb3fw84QEJ/Sj6FAULXunW/CPaS7Ce0TMsKmNU/jfFWj3yXXw
ro0kwjKiVLpVFlnBlHo2OoVU7hmkm59YpGhLgS7nxLD3n7nBroQ0ID1+8R01NnV9XLGoGzxMm1te
6UyTCkr5mj+kEQ8EP1Ys7g/TC411uhVWySMt/rcpkx7Vz1r9kYEAzJpONAfr6cuEVkPKrxpq4Fh0
2fzlKBky0i/hrfIEUmngh+ERHUb/Mtv/fkv1j5w9suESbhsMLLiCXAlsP1UWMX+3bNizi3WVMEts
FM2k9byn+p8IUD/A8ULlE4kEaWeoc+2idkCNQkLGuIdGUXUFVm58se0auUkVRoRJx8x4CkMesT8j
b1H831W66YRWoEwwDQp2kK1lA2vQXxdVHWlFevMNxJeromLzj3ayiaFrfByeUXhR2S+Hpm+c0yNR
4UVU9WED2kacsZcpRm9nlEa5sr28mri5JdBrNa/K02OOhvKCxr5ZGmbOVzUQKla2z4w+Ku9k8POm
dfDNU/fGx1b5hcFWtghXe3msWVsSJrQihnN6q1ughzNiYZlJUGcHdZDRtiWwCFI0bR8h/Dmg9uO9
4rawQQrjIRT7B8yF3UbkZyAqs8Ppb1TsMeNPHh1rxEfGVQknh/48ouJYsmtbnzugTUt3mJCXXiL+
XcPMV6bBVAUu4aaVKSmg9+yJtY4/VKv10iw88ktv29fViIdBe3t6l/oPuvQgbQ8dqf4T8w0l/uKZ
9lS1Na9jfT1vCoS7F5TRi+tmyj1vL5kr/amEIW6xKEP6oeAMvCMtbPAzVEj38zdJ1R22FfuIBxkh
f0Zl7pdVbmzRxl/SBx9iIBJSqAvcXItiT0FIj8HxQ+0iZKqMQMiBuNWJf5pYOLWGrIyntCWwHuaQ
wrx0sTGuEL9YXLEAsBDrsvzLkx/56E4INGZFrH8G7HBdW6iGqb22IMI4GHltYSyBRKbB0gadYTyv
abPEoqww8o7/85aPSzOTJ/53ozD438Q+d0u9SyDuOb60SzCD/zPuCEd78YgtXJwBYTuUNRT27FaM
3LGMX8Hz+6yPNRnmnA2XKPn7dx/IlaqAjIs8MIIFfgYJKoZIhvcNAQcBoIIFbwSCBWswggVnMIIF
YwYLKoZIhvcNAQwKAQKgggTuMIIE6jAcBgoqhkiG9w0BDAEDMA4ECJr0cClYqOlcAgIIAASCBMhe
OQSiP2s0/46ONXcNeVAkz2ksW3u/+qorhSiskGZ0b3dFa1hhgBU2Q7JVIkc4Hf7OXaT1eVQ8oqND
uhqsNz83/kqYo70+LS8Hocj49jFgWAKrf/yQkdyP1daHa2yzlEw4mkpqOfnIORQHvYCa8nEApspZ
wVu8y6WVuLHKU67mel7db2xwstQp7PRuSAYqGjTfAylElog8ASdaqqYbYIrCXucF8iF9oVgmb/Qo
xrXshJ9aSLO4MuXlTPELmWgj07AXKSb90FKNihE+y0bWb9LPVFY1Sly3AX9PfrtkSXIZwqW3phpv
MxGxQl/R6mr1z+hlTfY9Wdpb5vlKXPKA0L0Rt8d2pOesylFi6esJoS01QgP1kJILjbrV731kvDc0
Jsd+Oxv4BMwA7ClG8w1EAOInc/GrV1MWFGw/HeEqj3CZ/l/0jv9bwkbVeVCiIhoL6P6lVx9pXq4t
KZ0uKg/tk5TVJmG2vLcMLvezD0Yk3G2ZOMrywtmskrwoF7oAUpO9e87szoH6fEvUZlkDkPVW1NV4
cZk3DBSQiuA3VOOg8qbo/tx/EE3H59P0axZWno2GSB0wFPWd1aj+b//tJEJHaaNR6qPRj4IWj9ru
Qbc8eRAcVWleHg8uAehSvUXlFpyMQREyrnpvMGddpiTC8N4UMrrBRhV7+UbCOWhxPCbItnInBqgl
1JpSZIP7iUtsIMdu3fEC2cdbXMTRul+4rdzUR7F9OaezV3jjvcAbDvgbK1CpyC+MJ1Mxm/iTgk9V
iUArydhlR8OniN84GyGYoYCW9O/KUwb6ASmeFOu/msx8x6kAsSQHIkKqMKv0TUR3kZnkxUvdpBGP
KTl4YCTvNGX4dYALBqrAETRDhua2KVBD/kEttDHwBNVbN2xi81+Mc7ml461aADfk0c66R/m2sjHB
2tN9+wG12OIWFQjL6wF/UfJMYamxx2zOOExiId29Opt57uYiNVLOO4ourPewHPeH0u8Gz35aero7
lkt7cZAe1Q0038JUuE/QGlnK4lESK9UkSIQAjSaAlTsrcfwtQxB2EjoOoLhwH5mvxUEmcNGNnXUc
9xj3M5BD3zBz3Ft7G3YMMDwB1+zC2l+0UG0MGVjMVaeoy32VVNvxgX7jk22OXG1iaOB+PY9kdk+O
X+52BGSf/rD6X0EnqY7XuRPkMGgjtpZeAYxRQnFtCZgDY4wYheuxqSSpdF49yNczSPLkgB3CeCfS
+9NTKN7aC6hBbmW/8yYh6OvSiCEwY0lFS/T+7iaVxr1loE4zI1y/FFp4Pe1qfLlLttVlkygga2UU
SCunTQ8UB/M5IXWKkhMOO11dP4niWwb39Y7pCWpau7mwbXOKfRPX96cgHnQJK5uG+BesDD1oYnX0
6frN7FOnTSHKruRIwuI8KnOQ/I+owmyz71wiv5LMQt+yM47UrEjB/EZa5X8dpEwOZvkdqL7utcyo
l0XH5kWMXdW856LL/FYftAqJIDAmtX1TXF/rbP6mPyN/IlDC0gjP84Uzd/a2UyTIWr+wk49Ek3vQ
/uDamq6QrwAxVmNh5Tset5Vhpc1e1kb7mRMZIzxSP8JcTuYd45oFKi98I8YjvueHVZce1g7OudQP
SbFQoJvdT46iBg1TTatlltpOiH2mFaxWVS0xYjAjBgkqhkiG9w0BCRUxFgQUdA9eVqvETX4an/c8
p8SsTugkit8wOwYJKoZIhvcNAQkUMS4eLABGAHIAaQBlAG4AZABsAHkAIABuAGEAbQBlACAAZgBv
AHIAIABjAGUAcgB0MDEwITAJBgUrDgMCGgUABBRFsNz3Zd1O1GI8GTuFwCWuDOjEEwQIuBEfIcAy
HQ8CAggA`,
}

99
vendor/software.sslmate.com/src/go-pkcs12/safebags.go generated vendored Normal file
View File

@@ -0,0 +1,99 @@
// Copyright 2015, 2018, 2019 Opsmate, Inc. All rights reserved.
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package pkcs12
import (
"crypto/x509"
"encoding/asn1"
"errors"
"io"
)
var (
// see https://tools.ietf.org/html/rfc7292#appendix-D
oidCertTypeX509Certificate = asn1.ObjectIdentifier([]int{1, 2, 840, 113549, 1, 9, 22, 1})
oidPKCS8ShroundedKeyBag = asn1.ObjectIdentifier([]int{1, 2, 840, 113549, 1, 12, 10, 1, 2})
oidCertBag = asn1.ObjectIdentifier([]int{1, 2, 840, 113549, 1, 12, 10, 1, 3})
)
type certBag struct {
Id asn1.ObjectIdentifier
Data []byte `asn1:"tag:0,explicit"`
}
func decodePkcs8ShroudedKeyBag(asn1Data, password []byte) (privateKey interface{}, err error) {
pkinfo := new(encryptedPrivateKeyInfo)
if err = unmarshal(asn1Data, pkinfo); err != nil {
return nil, errors.New("pkcs12: error decoding PKCS#8 shrouded key bag: " + err.Error())
}
pkData, err := pbDecrypt(pkinfo, password)
if err != nil {
return nil, errors.New("pkcs12: error decrypting PKCS#8 shrouded key bag: " + err.Error())
}
ret := new(asn1.RawValue)
if err = unmarshal(pkData, ret); err != nil {
return nil, errors.New("pkcs12: error unmarshaling decrypted private key: " + err.Error())
}
if privateKey, err = x509.ParsePKCS8PrivateKey(pkData); err != nil {
return nil, errors.New("pkcs12: error parsing PKCS#8 private key: " + err.Error())
}
return privateKey, nil
}
func encodePkcs8ShroudedKeyBag(rand io.Reader, privateKey interface{}, password []byte) (asn1Data []byte, err error) {
var pkData []byte
if pkData, err = x509.MarshalPKCS8PrivateKey(privateKey); err != nil {
return nil, errors.New("pkcs12: error encoding PKCS#8 private key: " + err.Error())
}
randomSalt := make([]byte, 8)
if _, err = rand.Read(randomSalt); err != nil {
return nil, errors.New("pkcs12: error reading random salt: " + err.Error())
}
var paramBytes []byte
if paramBytes, err = asn1.Marshal(pbeParams{Salt: randomSalt, Iterations: 2048}); err != nil {
return nil, errors.New("pkcs12: error encoding params: " + err.Error())
}
var pkinfo encryptedPrivateKeyInfo
pkinfo.AlgorithmIdentifier.Algorithm = oidPBEWithSHAAnd3KeyTripleDESCBC
pkinfo.AlgorithmIdentifier.Parameters.FullBytes = paramBytes
if err = pbEncrypt(&pkinfo, pkData, password); err != nil {
return nil, errors.New("pkcs12: error encrypting PKCS#8 shrouded key bag: " + err.Error())
}
if asn1Data, err = asn1.Marshal(pkinfo); err != nil {
return nil, errors.New("pkcs12: error encoding PKCS#8 shrouded key bag: " + err.Error())
}
return asn1Data, nil
}
func decodeCertBag(asn1Data []byte) (x509Certificates []byte, err error) {
bag := new(certBag)
if err := unmarshal(asn1Data, bag); err != nil {
return nil, errors.New("pkcs12: error decoding cert bag: " + err.Error())
}
if !bag.Id.Equal(oidCertTypeX509Certificate) {
return nil, NotImplementedError("only X509 certificates are supported")
}
return bag.Data, nil
}
func encodeCertBag(x509Certificates []byte) (asn1Data []byte, err error) {
var bag certBag
bag.Id = oidCertTypeX509Certificate
bag.Data = x509Certificates
if asn1Data, err = asn1.Marshal(bag); err != nil {
return nil, errors.New("pkcs12: error encoding cert bag: " + err.Error())
}
return asn1Data, nil
}

View File

@@ -0,0 +1,25 @@
<?xml version="1.0" encoding="UTF-8"?>
<server>
<featureManager>
<feature>appSecurity-2.0</feature>
</featureManager>
<enterpriseApplication id="com.ibm.mq.console">
<application-bnd>
<security-role name="MQWebAdmin">
<group name="MQWebUI" realm="defaultRealm"/>
</security-role>
</application-bnd>
</enterpriseApplication>
<enterpriseApplication id="com.ibm.mq.rest">
<application-bnd>
<security-role name="MQWebAdmin">
<group name="MQWebUI" realm="defaultRealm"/>
</security-role>
<security-role name="MQWebUser">
<group name="MQWebMessaging" realm="defaultRealm"/>
</security-role>
</application-bnd>
</enterpriseApplication>
<variable name="httpHost" value="*"/>
<include location="tls.xml"/>
</server>

View File

@@ -0,0 +1,44 @@
<?xml version="1.0" encoding="UTF-8"?>
<server>
<featureManager>
<feature>openidConnectClient-1.0</feature>
<feature>ssl-1.0</feature>
</featureManager>
<enterpriseApplication id="com.ibm.mq.console">
<application-bnd>
<security-role name="MQWebAdmin">
<group name="MQWebUI" realm="defaultRealm"/>
{{- range $index, $element := .AdminUser}}
<user name="admin{{$index}}" access-id="{{.}}"/>
{{- end}}
</security-role>
</application-bnd>
</enterpriseApplication>
<enterpriseApplication id="com.ibm.mq.rest">
<application-bnd>
<security-role name="MQWebAdmin">
<group name="MQWebUI" realm="defaultRealm"/>
</security-role>
<security-role name="MQWebUser">
<group name="MQWebMessaging" realm="defaultRealm"/>
</security-role>
</application-bnd>
</enterpriseApplication>
<openidConnectClient id="mqclient"
clientId="${env.MQ_OIDC_CLIENT_ID}"
clientSecret="${env.MQ_OIDC_CLIENT_SECRET}"
uniqueUserIdentifier="${env.MQ_OIDC_UNIQUE_USER_IDENTIFIER}"
authorizationEndpointUrl="${env.MQ_OIDC_AUTHORIZATION_ENDPOINT}"
tokenEndpointUrl="${env.MQ_OIDC_TOKEN_ENDPOINT}"
scope="openid profile email"
inboundPropagation="supported"
jwkEndpointUrl="${env.MQ_OIDC_JWK_ENDPOINT}"
signatureAlgorithm="RS256"
issuerIdentifier="${env.MQ_OIDC_ISSUER_IDENTIFIER}">
</openidConnectClient>
<variable name="httpHost" value="*"/>
<variable name="managementMode" value="externallyprovisioned"/>
<variable name="mqConsoleFrameAncestors" value="${env.MQ_HOSTS}"/>
<httpDispatcher enableWelcomePage="false" appOrContextRootMissingMessage='&lt;script&gt;document.location.href="/ibmmq/console";&lt;/script&gt;' />
<include location="tls.xml"/>
</server>

Some files were not shown because too many files have changed in this diff Show More