test_pull_request_origin_extended_conformance_install_update failing across the board

[sjenning]

test_pull_request_origin_extended_conformance_install_update started failing across the board about 2hrs ago

https://openshift-gce-devel.appspot.com/builds/origin-ci-test/pr-logs/directory/test_pull_request_origin_extended_conformance_install_update

@simo5

[sjenning]

Fails here

Creating output directory: /tmp/tito
Tagging new version of openshift-ansible: 3.7.5-1 -> 3.7.6-1
Traceback (most recent call last):
  File "/usr/bin/tito", line 23, in <module>
    CLI().main(sys.argv[1:])
  File "/usr/lib/python2.7/site-packages/tito/cli.py", line 202, in main
    return module.main(argv)
  File "/usr/lib/python2.7/site-packages/tito/cli.py", line 666, in main
    return tagger.run(self.options)
  File "/usr/lib/python2.7/site-packages/tito/tagger/main.py", line 114, in run
    self._tag_release()
  File "/usr/lib/python2.7/site-packages/tito/tagger/main.py", line 136, in _tag_release
    self._check_tag_does_not_exist(self._get_new_tag(new_version))
  File "/usr/lib/python2.7/site-packages/tito/tagger/main.py", line 501, in _check_tag_does_not_exist
    raise Exception("Tag %s already exists!" % new_tag)
Exception: Tag openshift-ansible-3.7.6-1 already exists!
++ export status=FAILURE
++ status=FAILURE
+ set +o xtrace
########## FINISHED STAGE: FAILURE: BUILD AN OPENSHIFT-ANSIBLE RELEASE [00h 00m 02s] ##########

[smarterclayton]

@sdodson @michaelgugino @abutcher

[michaelgugino]

@smunilla Linking you to this issue.

[jupierce]

Hopefully fixed now. Branching 3.7 in openshift-ansible yesterday caused this. The normal CD build of 3.7 in the release-3.7 branch created the 3.7.6-1 tag. The openshift-ansible.spec in master was still at 3.7.4-1. I've bumped the master openshift-ansible.spec to 3.8 to avoid this.

[smarterclayton]

Install update is still broken, might be that openshift-ansible tag has the wrong release https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/17268/test_pull_request_origin_extended_conformance_install_update/9241/ On Nov 10, 2017, at 6:04 PM, Justin Pierce <notifications@github.com> wrote: Hopefully fixed now. Branching 3.7 in openshift-ansible yesterday caused this. The normal CD build of 3.7 in the release-3.7 branch created the 3.7.6-1 tag. The openshift-ansible.spec in master was still at 3.7.4-1. I've bumped the master openshift-ansible.spec to 3.8 to avoid this. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#17263 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABG_p26o9Kg4-mWnKHyT-CuteSSWQUyXks5s1NZdgaJpZM4QZ5YZ> .

[simo5]

Still broken on the next step:

######### STARTING STAGE: INSTALL ANSIBLE ATOMIC-OPENSHIFT-UTILS ##########
+ [[ -s /var/lib/jenkins/jobs/test_pull_request_origin_extended_conformance_install_update/workspace/activate ]]
+ source /var/lib/jenkins/jobs/test_pull_request_origin_extended_conformance_install_update/workspace/activate
++ export VIRTUAL_ENV=/var/lib/jenkins/origin-ci-tool/b4433dfdce6a5fba26d100d9416d78fd95716382
++ VIRTUAL_ENV=/var/lib/jenkins/origin-ci-tool/b4433dfdce6a5fba26d100d9416d78fd95716382
++ export PATH=/var/lib/jenkins/origin-ci-tool/b4433dfdce6a5fba26d100d9416d78fd95716382/bin:/sbin:/usr/sbin:/bin:/usr/bin
++ PATH=/var/lib/jenkins/origin-ci-tool/b4433dfdce6a5fba26d100d9416d78fd95716382/bin:/sbin:/usr/sbin:/bin:/usr/bin
++ unset PYTHON_HOME
++ export OCT_CONFIG_HOME=/var/lib/jenkins/jobs/test_pull_request_origin_extended_conformance_install_update/workspace/.config
++ OCT_CONFIG_HOME=/var/lib/jenkins/jobs/test_pull_request_origin_extended_conformance_install_update/workspace/.config
++ mktemp
+ script=/tmp/tmp.KroIrSOmRy
+ cat
+ chmod +x /tmp/tmp.KroIrSOmRy
+ scp -F ./.config/origin-ci-tool/inventory/.ssh_config /tmp/tmp.KroIrSOmRy openshiftdevel:/tmp/tmp.KroIrSOmRy
+ ssh -F ./.config/origin-ci-tool/inventory/.ssh_config -t openshiftdevel 'bash -l -c "timeout 600 /tmp/tmp.KroIrSOmRy"'
+ cd /data/src/github.com/openshift/aos-cd-jobs
+ pkg_name=origin
+ echo origin
+ echo 'openshift-ansible openshift-ansible-callback-plugins openshift-ansible-docs openshift-ansible-filter-plugins openshift-ansible-lookup-plugins openshift-ansible-playbooks openshift-ansible-roles'
++ cat OPENSHIFT_ANSIBLE_BUILT_VERSION
+ sudo python sjb/hack/determine_install_upgrade_version.py atomic-openshift-utils-3.8.1-1.git.0.d2eee04.el7.noarch --dependency_branch master
[ERROR] Can not determine install and upgrade version for the `atomic-openshift-utils` package
++ export status=FAILURE
++ status=FAILURE
+ set +o xtrace
########## FINISHED STAGE: FAILURE: INSTALL ANSIBLE ATOMIC-OPENSHIFT-UTILS [00h 01m 01s] ##########

[0xmichalis]

+ sudo python sjb/hack/determine_install_upgrade_version.py atomic-openshift-utils-3.8.1-1.git.0.d2eee04.el7.noarch --dependency_branch master
[ERROR] Can not determine install and upgrade version for the `atomic-openshift-utils` package

[stevekuznetsov]

Adding 3.7 repos to the test AMIs in openshift/origin-ci-tool@2453a17

Will need to build a new base AMI... which is currently blocked. Will figure out what we need to do there.

[stevekuznetsov]

New base AMIs blocked on #17268

[jhadvig]

FYI: The issue with the determine_install_upgrade_version.py is that we are building o-a with 3.8 tag

....       
atomic-openshift-utils.noarch                 3.5.145-1.git.0.e1e330f.el7                                                  rhel-7-server-ose-3.5-rpms            
atomic-openshift-utils.noarch                 3.6.8-1.git.0.8e26f8c.el7                                                    centos-paas-sig-openshift-origin36-rpms
atomic-openshift-utils.noarch                 3.6.153-1.el7                                                                centos-paas-sig-openshift-origin36-rpms
atomic-openshift-utils.noarch                 3.6.173.0.3-1.el7                                                            centos-paas-sig-openshift-origin36-rpms
atomic-openshift-utils.noarch                 3.6.173.0.60-1.el7                                                           centos-paas-sig-openshift-origin36-rpms
atomic-openshift-utils.noarch                 3.6.173.0.71-1.git.0.23eee8c.el7                                             rhel-7-server-ose-3.6-rpms            
atomic-openshift-utils.noarch                 3.8.1-1.git.0.0ef0a1e.el7                                                    openshift-ansible-local-release 

so we are missing the 3.7 which would be installed. That's why the script fails

[stevekuznetsov]

Right -- adding the repos for 3.7 in the commit above adds the 3.7 version that we need to be present

[0xmichalis]

Adding 3.7 repos to the test AMIs in openshift/origin-ci-tool@2453a17

Is there any documentation about places that need to change once we cut a new branch?