Tag Archives: cucm

Failure During Switch-Version: ERROR: Acquiring lock failed

Do you really want to switch between versions ?

Enter (yes/no)? yes
Switching Version and Restarting the Appliance …

Switch version duration can vary depending on the database size
and platform configuration. Please continue to monitor the
switchover process from here.
Waiting ……………………………………

Operation failed

ERROR: Acquiring lock failed

Easily solved with a simple reboot, then retry on the switch-version.

If this does fail, next steps would be:

  • Switch-version using Recovery CD
  • TAC case with Remote Account, and TAC to edit GRUB configuration file

 

#dontcalltac

Advertisements
Tagged , , , ,

VMware Tools 10.0 Update Failures Creates Outage – CSCux90747, CSCva09767, CSCuy84075

A friend hit this delightful defect this week:

 

 

There are several workarounds, but ultimately there’s a COP file for this – ciscocm.VMwareTools2016c.cop.sgn.  The attached defects cover several initial install and help-me-its-dead scenarios.

 

It’s a bit of a horror.  Watch out for those VMWare tools updates!

 

#dontcalltac

Tagged , , , , , , , , ,

CSCup36072 – CUCM/CUC Patch Upgrade Fails

Hit another horror defect today…  Patch upgrades apparently do not work as of 10.5.x and up!

 

Selection_041

 

The defect is described below:

 

 

The bug details are below – only workaround is to install with a bootable image!

 

Symptom:
When doing a fresh install of a Call Manager node, there is an option to use an upgrade patch option.

For CUCM 10.5, this approach cannot be used. Not even by using the bootable image for 10.0 and upgrade file for 10.5

Conditions:
Installing CUCM 10.5 with bootable image of a lower version and upgrade file of CUCM 10.5

Workaround:
None -> Use Bootable image for CUCM 10.5 itself

 

To resolve, the only “supported” method is to request a bootable image from TAC.

 

Terrible.  Terrible.  Terrible.

Tagged , , , , ,

CSCug20588 – Auto-Reg Failure – “Could not insert new row – duplicate value in a UN”

Had an issue that’s affected multiple deployments that I’ve been working on.  At one of my clients in particular, this reached an escalation level.  Just sharing this to assist others, as we went through 4 TAC Engineers to get a root cause and eventually this was attributed to a hidden Cisco defect in 10.x and above…  The first engineer told us that we needed to rebuild an 18 node CUCM cluster from scratch to fix this!

Just sharing this to assist others, as we went through 4 TAC Engineers to get a root cause and eventually this was attributed to a hidden Cisco defect in 10.x and above…  The first engineer told us that we needed to rebuild an 18 node CUCM cluster from scratch to fix this!  Don’t always believe TAC!

 

Problem Description

 

  • Client complains that auto-registered phones are failing to register
  • Eventually determined that the phone had been previously registered and what subsequently deleted
  • Certain clients say that if they manually configure the phone, it registers
  • Phone logs show a TFTP config error

 

Went on site and noted that there were persistent DB-related syslog alarms being generated.

 

The SDL logs showed the registration failure and a syslog event was logged:

 

00024541.012 |07:28:21.243 |AppInfo |SIPStationD(269) - checkDNsReceived: TotalCount=0
00024541.013 |07:28:21.243 |AppInfo |SIPStationD(269) - sendLineRegisterReq: DN match failed for AUTO-REG
00024541.014 |07:28:21.243 |AppInfo |SIPStationD(269) - sendLineRegisterReq: Phone is autoregistering...sending 200OK followed by a restart request
00024541.015 |07:28:21.243 |AppInfo |SIPStationD(269) - buildServiceControl: Sending callid from mPrimaryLineNum 0: 0UTqUn-9I4FebBfs1swQbo.GQ78HIxyf
00024541.016 |07:28:21.243 |AppInfo |SIPStationD(269) - sendServiceControlNotify: starting servicecontrol NOTIFY timer
00024541.017 |07:28:21.243 |AppInfo |SIPStationD(269) - sendServiceControlNotify: sending service-control NOTIFY
00024541.018 |07:28:21.243 |AppInfo |EndPointTransientConnection - An endpoint attempted to register but did not complete registration Connecting Port:5060 Device name:SEPAAAABBBBCCCC Device IP address:192.168.100.123 Device type:592 Reason Code:7 Protocol:SIP Device MAC address:AAAABBBBCCCC IPAddressAttributes:0 LastSignalReceived:SIPRegisterInd StationState:auto_register App ID:Cisco CallManager Cluster ID:MYCUCM Node ID:mycucmhostame
00024541.019 |07:28:21.243 |AlarmErr |AlarmClass: CallManager, AlarmName: EndPointTransientConnection, AlarmSeverity: Error, AlarmMessage: , AlarmDescription: An endpoint attempted to register but did not complete registration, AlarmParameters: ConnectingPort:5060, DeviceName:SEPAAAABBBBCCCC, IPAddress:192.168.100.123, DeviceType:592, Reason:7, Protocol:SIP, MACAddress:AAAABBBBCCCC, IPAddrAttributes:0, LastSignalReceived:SIPRegisterInd, StationState:auto_register, AppID:Cisco CallManager, ClusterID:MYCUCM, NodeID:mycucmhostame,
00024542.000 |07:28:21.243 |AppInfo |-->RISCMAccess::DeviceTransientConnection(...)
00024543.000 |07:28:21.243 |AppInfo |Device Transient deviceName : SEPAAAABBBBCCCC, IPAddress : 192.168.100.123, IPv6Address : not shown, IPv4Attribute :0, IPv6Attribute :0, Protocol : 2
00024544.000 |07:28:21.243 |AppInfo |DebugMsg deviceName : SEPAAAABBBBCCCC, DeviceType : 592, risClass: 1
00024545.000 |07:28:21.243 |AppInfo |<--RISCMAccess::DeviceTransientConnection(...)
00024541.020 |07:28:21.243 |AppInfo |SIPStationD(269) - Received register from Cisco phone for unconfigured line AUTO-REG; shutting down
00024541.021 |07:28:21.243 |AppInfo |StationTracking::updateMismatchCount: deviceId=SEPAAAABBBBCCCC, mismatchCount=174
00024541.022 |07:28:21.243 |AppInfo |SIPStationD(269) - processVersionStampMismatch: VersionStampMismatch detected, count = 174
00024541.023 |07:28:21.243 |AppInfo |SIPStationD(269) - DevStat-StopClose: LineRegisterReq send failure

 

 

And later we in the same trace see that CUCM sends a NOTIFY for the above:

 

NOTIFY sip:AUTO-REG@192.168.100.123:5060 SIP/2.0
Via: SIP/2.0/UDP 192.168.1.100:5060;branch=z9hG4bK2b1379d1903
From: <sip:192.168.1.100>;tag=789090571
To: <sip:AUTO-REG@192.168.100.123>
Call-ID: bb580e00-6df1b475-27b-247410ac@192.168.1.100
CSeq: 101 NOTIFY
Max-Forwards: 70
Date: Wed, 09 Mar 2016 05:28:21 GMT
User-Agent: Cisco-CUCM11.0
Event: service-control
Subscription-State: active
Contact: <sip:192.168.1.100:5060>
Content-Type: text/plain
Content-Length: 67

action=restart
RegisterCallId={0UTqUn-9I4FebBfs1swQbo.GQ78HIxyf}

 

I traced through the logs and saw that  soon after the NOTIFY was placed on the wire, the following was logged in SDL that highlighted the problem:

 

 

00024556.000 |07:28:21.385 |SdlSig |StationClose |wait |SIPHandler(1,100,80,1) |SIPStationInit(1,100,73,1) |1,100,10,1.591^192.168.100.123^* |[T:N-H:0,N:0,L:0,V:0,Z:0,D:0] CloseStationReason = 0 StationId = SEPAAAABBBBCCCC
00024546.004 |07:28:21.428 |AppInfo |DB: odbc::SQLException(Error fetching data from datasource: [Informix][Informix ODBC Driver][Informix]Could not insert new row - duplicate value in a UNIQUE INDEX column (Unique Index:assign).)
00024546.005 |07:28:21.429 |AppInfo |DBLException - An error occurred while performing database activities ErrorCode:-239 ExceptionString:Error fetching data from datasource: [Informix][Informix ODBC Driver][Informix]Could not insert new row - duplicate value in a UN App ID:Cisco CallManager Cluster ID:MYCUCM Node ID:mycucmhostame
00024546.006 |07:28:21.429 |AlarmErr |AlarmClass: CallManager, AlarmName: DBLException, AlarmSeverity: Alert, AlarmMessage: , AlarmDescription: An error occurred while performing database activities, AlarmParameters: ErrorCode:-239, ExceptionString:Error fetching data from datasource: [Informix][Informix ODBC Driver][Informix]Could not insert new row - duplicate value in a UN, AppID:Cisco CallManager, ClusterID:MYCUCM, NodeID:mycucmhostame,
00024546.007 |07:28:21.429 |AppInfo |dBProcs - ERROR odbc::SQLException addDevice()
00024546.008 |07:28:21.429 |AppInfo |ProcessDb - ERROR initializing_DbStationAutoRegisterReq - addDevice failed

 

So my initial thought was a bug with orphaned data.  Had to be something along this line as the SDL error was referencing a duplicate.  So , I relunctantly decided to go straight to  TAC for analysis.   I sent this off to TAC and got crickets…  The assigned engineer eventually told me this was an NTP issue which didn’t make sense at all.

Another colleague logged a similar case that we had at another client.  2 re-queues later we got a great engineer who identified  CSCug20588 as the cause of our problem.  It’s a hidden TAC case.

 

The fix?  Well, a RIDICULOUS bug!

 

When auto-registering a phone in 10.x the Auto-Reg config uses a UDT/ULT.  This results in a phone-specific Phone Button Template being created when a phone is inserted into the Informix database, of the form:

 

SEP<MAC-ADDRESS>-SIP-Individual Template

 

The damn PBT was causing the phone not to register!  This obvoiusly remained in the list of PBT’s when the phone was deleted from CUCM.  When it came to AUTO-REG the phone, CUCM clearly does not look to delete/re-add the PBT or simply use the existing one.  Thus the DB insertion error, and the problematic behaviour.

I deleted the PBT and immediately the phone registered.

There’s no fix for this in 10.x and we’re waiting on the BU to confirm an official 11.x SU that will contain the fix for this.  One to watch out for!

 

Tagged , , , ,

Interpreting MGCP Debug Parameters

I manage a number of deployments that still feature ISDN extensively, so debug mgcp packet is something that I’m reading off IOS CLI on a fairly routine basis at the moment.

There are a lot of blog posts that cover reading MGCP debugs pretty well, but one thing that I struggled with for a while was being able to quickly interpret the  debug parameters and what they stood for in each respective MGCP message.  Often these are fairly intuitive, but sometimes a quick point of reference is needed.

I looked around on Cisco forums etc., and really there just isn’t anything that covers this well.  Eventually I resorted to the RFC, and was pleasantly unsurprised that all the parameters were well tabulated and easily reference-able.

I’ve included the list below for reference, but as always it is best to go straight to RFC 3435 to get this information!

 


3.2.2 Parameter Lines

Parameter lines are composed of a parameter name, which in most cases
   is composed of one or two characters, followed by a colon, optional
   white space(s) and the parameter value.  The parameters that can be
   present in commands are defined in the following table:


    ------------------------------------------------------------------
   |Parameter name        | Code |  Parameter value                   |
   |----------------------|------|------------------------------------|
   |BearerInformation     |   B  |  See description (3.2.2.1).        |
   |CallId                |   C  |  See description (3.2.2.2).        |
   |Capabilities          |   A  |  See description (3.2.2.3).        |
   |ConnectionId          |   I  |  See description (3.2.2.5).        |
   |ConnectionMode        |   M  |  See description (3.2.2.6).        |
   |ConnectionParameters  |   P  |  See description (3.2.2.7).        |
   |DetectEvents          |   T  |  See description (3.2.2.8).        |
   |DigitMap              |   D  |  A text encoding of a digit map.   |
   |EventStates           |   ES |  See description (3.2.2.9).        |
   |LocalConnectionOptions|   L  |  See description (3.2.2.10).       |
   |MaxMGCPDatagram       |   MD |  See description (3.2.2.11).       |
   |NotifiedEntity        |   N  |  An identifier, in RFC 821 format, |
   |                      |      |  composed of an arbitrary string   |
   |                      |      |  and of the domain name of the     |
   |                      |      |  requesting entity, possibly com-  |
   |                      |      |  pleted by a port number, as in:   |
   |                      |      |    Call-agent@ca.example.net:5234  |
   |                      |      |  See also Section 3.2.1.3.         |
   |ObservedEvents        |   O  |  See description (3.2.2.12).       |
   |PackageList           |   PL |  See description (3.2.2.13).       |
   |QuarantineHandling    |   Q  |  See description (3.2.2.14).       |
   |ReasonCode            |   E  |  A string with a 3 digit integer   |
   |                      |      |  optionally followed by a set of   |
   |                      |      |  arbitrary characters (3.2.2.15).  |
   |RequestedEvents       |   R  |  See description (3.2.2.16).       |
   |RequestedInfo         |   F  |  See description (3.2.2.17).       |
   |RequestIdentifier     |   X  |  See description (3.2.2.18).       |
   |ResponseAck           |   K  |  See description (3.2.2.19).       |
   |RestartDelay          |   RD |  A number of seconds, encoded as   |
   |                      |      |  a decimal number.                 |
   |RestartMethod         |   RM |  See description (3.2.2.20).       |
   |SecondConnectionId    |   I2 |  Connection Id.                    |
   |SecondEndpointId      |   Z2 |  Endpoint Id.                      |
   |SignalRequests        |   S  |  See description (3.2.2.21).       |
   |SpecificEndPointId    |   Z  |  An identifier, in RFC 821 format, |
   |                      |      |  composed of an arbitrary string,  |
   |                      |      |  followed by an "@" followed by    |
   |                      |      |  the domain name of the gateway to |
   |                      |      |  which this endpoint is attached.  |
   |                      |      |  See also Section 3.2.1.3.         |
   |RemoteConnection-     |   RC |  Session Description.              |
   |         Descriptor   |      |                                    |
   |LocalConnection-      |   LC |  Session Description.              |
   |         Descriptor   |      |                                    |
    ------------------------------------------------------------------

   The parameters are not necessarily present in all commands.  The
   following table provides the association between parameters and
   commands.  The letter M stands for mandatory, O for optional and F
   for forbidden.  Unless otherwise specified, a parameter MUST NOT be
   present more than once.

This table is great as I can simply look up the parameter code in my MGCP packet debug and do a quick interpretation.  I also find this very handy when reading off MGCP Connection ID’s and other call-specific tracing that can be cross-referenced in SDL!


 

For example, in a simple AUEP (Audit Endpoint), we see:

 

*Jul 17 11:14:18.332: MGCP Packet received from 192.168.123.213:2427—>
AUEP 31 S0/SU2/DS1-0/1@RTR1.uc.lab MGCP 0.1
F: X, A, I
<—

What has remote side requested information on?

 

Firstly, the side has requested information in a comma-separated list:

|RequestedInfo         |   F  |

 

The list contains the following:

|RequestIdentifier     |   X  |
|Capabilities          |   A  |
|ConnectionId          |   I  |

 

Simple stuff.

 

#dontcalltac

Tagged , , , , , , ,

CUCM Video Codec Preferences Support – CSCuw53802

Colleague called me for assistance with a video call that wouldn’t set up when dialling in to a VC bridge.

The call flow was from a Polycom endpoint to the new Conductor-managed vTS bridge that I’d installed a week before.  The endpoint could dial into the Rendezvous VC bridge, but calls only had audio.  Some additional testing showed that this wasn’t working for point-to-point calls to Jabber as well, which helped us rule out the bridge completely.

My colleague pulled a Wireshark from Jabber, and the signalling confirmed a codec support issue on the Polycom:

Jabber INVITE with H.264:

Selection_035

However, the response clearly showed that our Polycom unit didn’t like this, and marked the video streams as inactive:

Selection_036

My colleague did some reading up on our somewhat outdated Polycom unit, and we ultimately found that some release keys and a firmware upgrade would be needed.  Customer not too happy, but nothing to do here other than upgrade…

However, my colleague did post a question to me:

Can we not just set some video codec preferences for this call, or at least for the call into the VC bridge?

Well, the answer is… no!

 

As of 11.0 this is not supported.  However, thankfully this does appear to have been considered in the roadmap for CUCM 11.5:

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCuw53802/

Selection_037

 

This is good news, as the CUCM-based video architecture is now quite mature, having gone through several iterations since it’s inception from CUCM 9.x.

As the market for Skype for Business grows, H.265 begins to create more and more headaches due to patent licensing problems, and Cisco subsequently begins to punt Thor as a potential future video standard, this new feature should hopefully solve some problems down the line.

 

#dontcalltac

Tagged , , , , , , , , , , , , , , ,

Accessing the jabber-config.xml Jabber Configuration File

Often when working on Cisco Jabber installations, you will be required to edit an existing jabber-config.xml file to change client parameters or support new functionality, or simply as part of an audit or system assessment want to have a look at what customization is being used for Jabber already.

There are several ways to access the jabber-config.xml file from your CUCM/TFTP server or Jabber client PC.

When troubleshooting, it is often helpful to know the methods to view from both the TFTP server and the Jabber client PC.  Discrepancies can be a good starting point for troubleshooting.

Method 1 – CUOS

From CUOS, run the following command:

admin:file view tftp jabber-config.xml

 

Method 2 – Browser

From your browser, enter the following:

http://X.X.X.X:6970/jabber-config.xml

where X.X.X.X is the CUCM (i.e. TFTP Server) IP Address

 

Method 3 – TFTP Client

Use a TFTP Client such as Windows TFTP client or TFTPD32 and GET the jabber-config.xml file.

 

Method 4 – Windows Client PC

Fetch this from the following path:

%USERPROFILE%\AppData\Roaming\Cisco\Unified Communications\Jabber\CSF\Config

You may need to enable showing of hidden files on the Windows desktop to reach this.

 

If you know of more, please let me know!

#dontcalltac

Tagged , , , , , ,

Cisco Support Community Collaboration Videos

Thanks to Java for sharing this – can’t believe I’ve never come across this!

A really, really awesome resource for a number of Collab-related support topics.  Some excellent videos on topics such as:

  • CUOS certificates
  • Self-Provisioning
  • Conference Now
  • PCD
  • TelePresence
  • VCS
  • Service Discovry and UDS
  • Various phone model factory reset procedures

 

 

Enjoy! 🙂

Tagged , , , , , , , , , ,

CUOS Certificate Expiration Issues and Regeneration

I arrived on site at a remote client picking up a deployment that was done some 18 months ago for additional scope of work.  However, when I arrived to begin, I encountered some strange behaviour that I struggled to attribute to anything obvious.

What I saw was:

 

  • High number of AXL connections on CUCM:

admin:utils diagnose test

Log file: platform/log/diag3.log

Starting diagnostic test(s)
===========================
test – disk_space : Passed (available: 1796 MB, used: 12360 MB)
skip – disk_files : This module must be run directly and off hours
test – service_manager : Passed
test – tomcat : Passed
test – tomcat_deadlocks : Passed
test – tomcat_keystore : Passed
test – tomcat_connectors : Passed
test – tomcat_threads : Passed
test – tomcat_memory : Passed
test – tomcat_sessions : FailedThe following web applications have an unusually large number of active sessions: axl. Please collect all of the Tomcat logs for root cause analysis: file get activelog tomcat/logs/*
skip – tomcat_heapdump : This module must be run directly and off hours
test – validate_network : Passed
test – raid : Passed
test – system_info : Passed (Collected system information in diagnostic log)
test – ntp_reachability : Passed
test – ntp_clock_drift : Passed
test – ntp_stratum : Passed
skip – sdl_fragmentation : This module must be run directly and off hours
skip – sdi_fragmentation : This module must be run directly and off hours

 

  • Backups Failing

 

  • Jabber services listed as” UNKNOWN” from the GUI, even though listed as “STARTED” from Serviceability and from CLI:

 

Selection_006

 

  • Immediate Switch-Version failed after an upgrade due to SELinux failing to upgrade correctly, causing differing behaviour on different servers:
    – Java errors on login to server and CLI scripts unable to run due to permissions issues
    – A Cisco DB service refusing to start!
    – Kicked out of ssh session directly after authentication – SELinux issue

 

 

The SELinux issue actually prevented a switch-back as this could not be initiated from CLI.  To resolve this I had to complete a manual partition switch using the CUCM/CUC Recovery CD as well as a File System Check and Repair!

 

This was all very, very strange!

 

Once back to a “stable” platform after the partition swap and file-system check, I again attempted an upgrade, which this time failed:

04/20/2016 19:46:49 upgrade_install.sh|Started auditd…|<LVL::Info>
04/20/2016 19:46:49 upgrade_install.sh|Started setroubleshoot…|<LVL::Info>
04/20/2016 19:46:49 upgrade_install.sh|Changed selinux mode to enforcing|<LVL::Info>
04/20/2016 19:46:49 upgrade_install.sh|Cleaning up rpm_archive…|<LVL::Info>
04/20/2016 19:46:49 upgrade_install.sh|Removing /common/rpm-archive/10.5.2.12901-1|<LVL::Info>
04/20/2016 19:46:50 upgrade_install.sh|File:/usr/local/bin/base_scripts/upgrade_install.sh:604, Function: main(), Upgrade Failed — (1)|<LVL::Error>
04/20/2016 19:46:50 upgrade_install.sh|Parse argument status=upgrade.stage.error|<LVL::Debug>
04/20/2016 19:46:50 upgrade_install.sh|_set_upgrade_status_attribute: status set to upgrade.stage.error|<LVL::Debug>
04/20/2016 19:46:50 upgrade_install.sh|is_upgrade_lock_available: Upgrade lock is not available.|<LVL::Debug>
04/20/2016 19:46:50 upgrade_install.sh|is_upgrade_in_progress: Already locked by this process (pid: 32606).|<LVL::Debug>
04/20/2016 19:46:50 upgrade_install.sh|release_upgrade_lock: Releasing lock (pid: 32606)|<LVL::Debug>

 

This time, I had selected an immediate switch-version instead of using a 2-stage process after a successful upgrade.  Again, I could confirm a successful upgrade:

 

04/20/2016 19:39:01 post_upgrade|Post Upgrade RTMTFinish|<LVL::Notice>
04/20/2016 19:39:01 post_upgrade|========================= Upgrade complete. Awaiting switch to version. =========================|<LVL::Info>
04/20/2016 19:39:01 upgrade_manager.sh|Post-upgrade processing complete|<LVL::Info>
04/20/2016 19:39:01 upgrade_manager.sh|Application install on inactive partition complete|<LVL::Info>
04/20/2016 19:39:01 upgrade_manager.sh|L2 upgrade… Run selinux_on_inactive_partition|<LVL::Info>

 

So now, I concentrated on the post-upgrade switch-version logging, and picked up on this:

 

04/20/2016 19:46:38 upgrade_manager.sh|(CAPTURE) /sbin/restorecon.orig set context /etc/opt/cisco/elm/client/.security/trust_certs/Cisco_Root_CA_M1.pem->system_u:object_r:cisco_etc_t:s0 failed:’Operation not permitted’|<LVL::Debug>
04/20/2016 19:46:38 upgrade_manager.sh|(CAPTURE) /sbin/restorecon.orig set context /etc/opt/cisco/elm/client/.security/trust_certs/Cisco_Root_CA_M1.der->system_u:object_r:cisco_etc_t:s0 failed:’Operation not permitted’|<LVL::Debug>
04/20/2016 19:46:38 upgrade_manager.sh|(CAPTURE) /sbin/restorecon.orig set context /home/sftpuser/sftp_connect.sh->specialuser_u:object_r:user_home_t:s0 failed:’Operation not permitted’|<LVL::Debug>

 

It was then that I became very confused.  These strange issues together all pointed to **certificate expiration**.  However, the system had been installed less than 18 months ago, so this didn’t make sense!  I checked the certificates and confirmed – all expired.  Then it dawned on me.  When we had installed the system, we had staged the network and voice at the same time.  We’d pointed NTP to the firewall – which must have, prior to go-live, been set as a master by our firewall team … with the wrong date-time! 😦

So, root cause found, but now to resolve?!

 

Certificates are a tricky business in CUCM since the implementation of Security by Default.  I don’t intend to cover this topic here (it’s absolutely huge), but I will say this – changing certificates on CUCM can run the risk of getting you fired.

If you don’t know 100% what you are doing, you could cause an outage that will require serious skill to repair, an emergency TAC engagement,  manual phone intervention or a combination of the 3.

 

So, I now needed to complete Certificate Regeneration on CUCM, CUC and IM&P.

Some notes here:

  • My cluster was not in Mixed Mode – my life just got a whole lot easier!
  • The client didn’t have a Private CA, so we were forced to use Self-Signed.  A major consideration if you have signed certs.
  • I was working on a BE6K – with no redundancy – so for me…  ITL files are going to be an issue.  I must to do a “Pre- 8.0 Rollback” and blank all the ITL files before I start.
  • Multi-App integrations are probably going to require manual certificate download/import between clusters for IM&P

 

Following the above process, I carefully:

  1. Blanked all ITL files, with necessary TVS and TFTP service restarts as required
  2. Stopped TFTP Services
  3. Regenerated all applicable certificates as per the above guide
  4. Manually deleted “-trust” certificates as per the guide – I preferred the GUI to the CLI for this – watch out for related bugs in the above guide.
  5. Restarted all applicable services
  6. Removed Pre-8.0 Rollback Enterprise Parameter ITL blanking
  7. Again, restarted TVS and TFTP services
  8. Imported regenerated certs for intra- and inter-cluster communications

 

An Important Note:

When restart TFTP, always wait 5-10mins for the TFTP files to rebuild on the server.  More haste, Less-Job-Come-Monday.

 

Rebooted IM&P, and immediately saw that all the affected IM&P services were listed correctly:

Selection_010.png

 

Helpful links on SBD (Security by Default) and CUOS Certificates in General:

 

 

Check out the ITLRecovery enhancements in 10.x and above!

 

#dontcalltac

Tagged , , , , , ,

Gathering CUOS Installation Information Prior to Upgrade

It may sound obvious, but prior to any upgrade – prepare for the eventuality of a system rebuild.  I hit an upgrade today that required both a Recovery CD partition swap and file system check on the original Active Partition to recover the system, after SELinux failures occurred after a switch-version.

For a rebuild, collect the following from CLI prior to upgrade:

  1. show network eth0 

  2. show status

  3. show version active

  4. utils ntp config

  5. show web-security

 

This will collect IP/Hostname, DNS, NTP/Timezone, firmware and certificate details(last one is critical!).

This is all negated if you have Answer Files set up to begin with and/or are using PCD.

 

HTH

Tagged , , ,
Collaboration Engineer

All things Collaboration - Posts to save for when you need them

Gerry Keleghan's Blog

A Blog about Cisco Unified Communications

ccieme

my personal journey to ccie collaboration

Striving for greatness

Thoughts on emerging tech, open source, and life

Network Experts Blog

“Knowledge comes by eyes always open and working hands.”

SIP Adventures

A unified communications blog by Andrew Prokop

The Cloverhound Blog

Cloverhound Employees Talk Unified Communications and Contact Center

Warcop

Fog navigator. Get out of the clouds. Down to earth solutions. @Warcop

Cisco Collab Engineering Tips

Michael White - CCIE #26626

Darkroomstory

Photography by Manos,

afterthenumber

Thoughts and experiences of a Cisco Collaboration engineer after clearing the CCIE lab...

Longreads

The best longform stories on the web

The Daily Post

The Art and Craft of Blogging

The WordPress.com Blog

The latest news on WordPress.com and the WordPress community.