After the installation of a brand new SCOM environment one of the lasts steps before handing the system over to the production was to ensure, that the SCOM Console is available for all the users via Network Load Balancing address in both Web and thick client versions. The Active Directory team set up appropriate DNS entry and the Network team set up the load balancing....
After the configuration was completed it turned out, that even though the web console was working properly the application one behaved very strange. It was connecting properly from any place to the direct servers' FQDN addresses without any authentication prompt, but when directed to the NLB address it didn't recognize the user account and was asking for credentials. When provided with the valid ones it was throwing the following error:
Operations Manager Console error when connecting via NLB address
The whole output of the error message is below:
====================================================
Date: 12/8/2016 5:19:09 PM
Application: Operations Manager
Application Version: 7.0.9538.1136
Severity: Error
Message: Failed to connect to server 'XXX'
Microsoft.EnterpriseManagement.Common.ServerDisconnectedException: The client has been disconnected from the server. Please call ManagementGroup.Reconnect() to reestablish the connection. ---> System.ServiceModel.ProtocolException: You have tried to create a channel to a service that does not support .Net Framing. It is possible that you are encountering an HTTP endpoint. ---> System.IO.InvalidDataException: Expected record type 'PreambleAck', found '72'.
--- End of inner exception stack trace ---
Server stack trace:
at System.ServiceModel.Channels.FramingDecoder.ValidatePreambleAck(FramingRecordType foundType)
at System.ServiceModel.Channels.ClientDuplexDecoder.Decode(Byte[] bytes, Int32 offset, Int32 size)
at System.ServiceModel.Channels.ConnectionUpgradeHelper.ValidateUpgradeResponse(Byte[] buffer, Int32 count, ClientFramingDecoder decoder)
at System.ServiceModel.Channels.ConnectionUpgradeHelper.InitiateUpgrade(StreamUpgradeInitiator upgradeInitiator, IConnection& connection, ClientFramingDecoder decoder, IDefaultCommunicationTimeouts defaultTimeouts, TimeoutHelper& timeoutHelper)
at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.SendPreamble(IConnection connection, ArraySegment`1 preamble, TimeoutHelper& timeoutHelper)
at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.DuplexConnectionPoolHelper.AcceptPooledConnection(IConnection connection, TimeoutHelper& timeoutHelper)
at System.ServiceModel.Channels.ConnectionPoolHelper.EstablishConnection(TimeSpan timeout)
at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.CallOnceManager.CallOnce(TimeSpan timeout, CallOnceManager cascade)
at System.ServiceModel.Channels.ServiceChannel.EnsureOpened(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Microsoft.EnterpriseManagement.Common.Internal.IDispatcherService.Connect(SdkClientConnectionOptions connectionOptions)
at Microsoft.EnterpriseManagement.Common.Internal.SdkDataLayerProxyCore.Initialize(EnterpriseManagementConnectionSettings connectionSettings, SdkChannelObject`1 channelObjectDispatcherService)
at Microsoft.EnterpriseManagement.Common.Internal.SdkDataLayerProxyCore.CreateEndpoint[T](EnterpriseManagementConnectionSettings connectionSettings, SdkChannelObject`1 channelObjectDispatcherService)
--- End of inner exception stack trace ---
at Microsoft.EnterpriseManagement.Common.Internal.ExceptionHandlers.HandleChannelExceptions(Exception ex)
at Microsoft.EnterpriseManagement.Common.Internal.SdkDataLayerProxyCore.CreateEndpoint[T](EnterpriseManagementConnectionSettings connectionSettings, SdkChannelObject`1 channelObjectDispatcherService)
at Microsoft.EnterpriseManagement.Common.Internal.SdkDataLayerProxyCore.ConstructEnterpriseManagementGroupInternal[T,P](EnterpriseManagementConnectionSettings connectionSettings, ClientDataAccessCore clientCallback)
at Microsoft.EnterpriseManagement.Common.Internal.SdkDataLayerProxyCore.RetrieveEnterpriseManagementGroupInternal[T,P](EnterpriseManagementConnectionSettings connectionSettings, ClientDataAccessCore callbackDispatcherService)
at Microsoft.EnterpriseManagement.Common.Internal.SdkDataLayerProxyCore.Connect[T,P](EnterpriseManagementConnectionSettings connectionSettings, ClientDataAccessCore callbackDispatcherService)
at Microsoft.EnterpriseManagement.ManagementGroup.InternalInitialize(EnterpriseManagementConnectionSettings connectionSettings, ManagementGroupInternal internals)
at Microsoft.EnterpriseManagement.Mom.Internal.UI.Common.ManagementGroupSessionManager.Connect(String server, String username, SecureString password, String domain)
at Microsoft.EnterpriseManagement.Monitoring.Console.Internal.ConsoleWindowBase.ConnectWithCredentials(Exception ex, ConsoleJobEventArgs args)
System.ServiceModel.ProtocolException: You have tried to create a channel to a service that does not support .Net Framing. It is possible that you are encountering an HTTP endpoint. ---> System.IO.InvalidDataException: Expected record type 'PreambleAck', found '72'.
--- End of inner exception stack trace ---
Server stack trace:
at System.ServiceModel.Channels.FramingDecoder.ValidatePreambleAck(FramingRecordType foundType)
at System.ServiceModel.Channels.ClientDuplexDecoder.Decode(Byte[] bytes, Int32 offset, Int32 size)
at System.ServiceModel.Channels.ConnectionUpgradeHelper.ValidateUpgradeResponse(Byte[] buffer, Int32 count, ClientFramingDecoder decoder)
at System.ServiceModel.Channels.ConnectionUpgradeHelper.InitiateUpgrade(StreamUpgradeInitiator upgradeInitiator, IConnection& connection, ClientFramingDecoder decoder, IDefaultCommunicationTimeouts defaultTimeouts, TimeoutHelper& timeoutHelper)
at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.SendPreamble(IConnection connection, ArraySegment`1 preamble, TimeoutHelper& timeoutHelper)
at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.DuplexConnectionPoolHelper.AcceptPooledConnection(IConnection connection, TimeoutHelper& timeoutHelper)
at System.ServiceModel.Channels.ConnectionPoolHelper.EstablishConnection(TimeSpan timeout)
at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.CallOnceManager.CallOnce(TimeSpan timeout, CallOnceManager cascade)
at System.ServiceModel.Channels.ServiceChannel.EnsureOpened(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Microsoft.EnterpriseManagement.Common.Internal.IDispatcherService.Connect(SdkClientConnectionOptions connectionOptions)
at Microsoft.EnterpriseManagement.Common.Internal.SdkDataLayerProxyCore.Initialize(EnterpriseManagementConnectionSettings connectionSettings, SdkChannelObject`1 channelObjectDispatcherService)
at Microsoft.EnterpriseManagement.Common.Internal.SdkDataLayerProxyCore.CreateEndpoint[T](EnterpriseManagementConnectionSettings connectionSettings, SdkChannelObject`1 channelObjectDispatcherService)
System.IO.InvalidDataException: Expected record type 'PreambleAck', found '72'.
Application: Operations Manager
Application Version: 7.0.9538.1136
Severity: Error
Message: Failed to connect to server 'XXX'
Microsoft.EnterpriseManagement.Common.ServerDisconnectedException: The client has been disconnected from the server. Please call ManagementGroup.Reconnect() to reestablish the connection. ---> System.ServiceModel.ProtocolException: You have tried to create a channel to a service that does not support .Net Framing. It is possible that you are encountering an HTTP endpoint. ---> System.IO.InvalidDataException: Expected record type 'PreambleAck', found '72'.
--- End of inner exception stack trace ---
Server stack trace:
at System.ServiceModel.Channels.FramingDecoder.ValidatePreambleAck(FramingRecordType foundType)
at System.ServiceModel.Channels.ClientDuplexDecoder.Decode(Byte[] bytes, Int32 offset, Int32 size)
at System.ServiceModel.Channels.ConnectionUpgradeHelper.ValidateUpgradeResponse(Byte[] buffer, Int32 count, ClientFramingDecoder decoder)
at System.ServiceModel.Channels.ConnectionUpgradeHelper.InitiateUpgrade(StreamUpgradeInitiator upgradeInitiator, IConnection& connection, ClientFramingDecoder decoder, IDefaultCommunicationTimeouts defaultTimeouts, TimeoutHelper& timeoutHelper)
at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.SendPreamble(IConnection connection, ArraySegment`1 preamble, TimeoutHelper& timeoutHelper)
at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.DuplexConnectionPoolHelper.AcceptPooledConnection(IConnection connection, TimeoutHelper& timeoutHelper)
at System.ServiceModel.Channels.ConnectionPoolHelper.EstablishConnection(TimeSpan timeout)
at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.CallOnceManager.CallOnce(TimeSpan timeout, CallOnceManager cascade)
at System.ServiceModel.Channels.ServiceChannel.EnsureOpened(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Microsoft.EnterpriseManagement.Common.Internal.IDispatcherService.Connect(SdkClientConnectionOptions connectionOptions)
at Microsoft.EnterpriseManagement.Common.Internal.SdkDataLayerProxyCore.Initialize(EnterpriseManagementConnectionSettings connectionSettings, SdkChannelObject`1 channelObjectDispatcherService)
at Microsoft.EnterpriseManagement.Common.Internal.SdkDataLayerProxyCore.CreateEndpoint[T](EnterpriseManagementConnectionSettings connectionSettings, SdkChannelObject`1 channelObjectDispatcherService)
--- End of inner exception stack trace ---
at Microsoft.EnterpriseManagement.Common.Internal.ExceptionHandlers.HandleChannelExceptions(Exception ex)
at Microsoft.EnterpriseManagement.Common.Internal.SdkDataLayerProxyCore.CreateEndpoint[T](EnterpriseManagementConnectionSettings connectionSettings, SdkChannelObject`1 channelObjectDispatcherService)
at Microsoft.EnterpriseManagement.Common.Internal.SdkDataLayerProxyCore.ConstructEnterpriseManagementGroupInternal[T,P](EnterpriseManagementConnectionSettings connectionSettings, ClientDataAccessCore clientCallback)
at Microsoft.EnterpriseManagement.Common.Internal.SdkDataLayerProxyCore.RetrieveEnterpriseManagementGroupInternal[T,P](EnterpriseManagementConnectionSettings connectionSettings, ClientDataAccessCore callbackDispatcherService)
at Microsoft.EnterpriseManagement.Common.Internal.SdkDataLayerProxyCore.Connect[T,P](EnterpriseManagementConnectionSettings connectionSettings, ClientDataAccessCore callbackDispatcherService)
at Microsoft.EnterpriseManagement.ManagementGroup.InternalInitialize(EnterpriseManagementConnectionSettings connectionSettings, ManagementGroupInternal internals)
at Microsoft.EnterpriseManagement.Mom.Internal.UI.Common.ManagementGroupSessionManager.Connect(String server, String username, SecureString password, String domain)
at Microsoft.EnterpriseManagement.Monitoring.Console.Internal.ConsoleWindowBase.ConnectWithCredentials(Exception ex, ConsoleJobEventArgs args)
System.ServiceModel.ProtocolException: You have tried to create a channel to a service that does not support .Net Framing. It is possible that you are encountering an HTTP endpoint. ---> System.IO.InvalidDataException: Expected record type 'PreambleAck', found '72'.
--- End of inner exception stack trace ---
Server stack trace:
at System.ServiceModel.Channels.FramingDecoder.ValidatePreambleAck(FramingRecordType foundType)
at System.ServiceModel.Channels.ClientDuplexDecoder.Decode(Byte[] bytes, Int32 offset, Int32 size)
at System.ServiceModel.Channels.ConnectionUpgradeHelper.ValidateUpgradeResponse(Byte[] buffer, Int32 count, ClientFramingDecoder decoder)
at System.ServiceModel.Channels.ConnectionUpgradeHelper.InitiateUpgrade(StreamUpgradeInitiator upgradeInitiator, IConnection& connection, ClientFramingDecoder decoder, IDefaultCommunicationTimeouts defaultTimeouts, TimeoutHelper& timeoutHelper)
at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.SendPreamble(IConnection connection, ArraySegment`1 preamble, TimeoutHelper& timeoutHelper)
at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.DuplexConnectionPoolHelper.AcceptPooledConnection(IConnection connection, TimeoutHelper& timeoutHelper)
at System.ServiceModel.Channels.ConnectionPoolHelper.EstablishConnection(TimeSpan timeout)
at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout)
at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.CallOnceManager.CallOnce(TimeSpan timeout, CallOnceManager cascade)
at System.ServiceModel.Channels.ServiceChannel.EnsureOpened(TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Microsoft.EnterpriseManagement.Common.Internal.IDispatcherService.Connect(SdkClientConnectionOptions connectionOptions)
at Microsoft.EnterpriseManagement.Common.Internal.SdkDataLayerProxyCore.Initialize(EnterpriseManagementConnectionSettings connectionSettings, SdkChannelObject`1 channelObjectDispatcherService)
at Microsoft.EnterpriseManagement.Common.Internal.SdkDataLayerProxyCore.CreateEndpoint[T](EnterpriseManagementConnectionSettings connectionSettings, SdkChannelObject`1 channelObjectDispatcherService)
System.IO.InvalidDataException: Expected record type 'PreambleAck', found '72'.
====================================================
Reason:
The error message is very uncommon for SCOM configuration and I haven't found any trace of the similar error occurring in regards to the SCOM Console application. There is no further need to any configuration on the SCOM side of things in order to set up Console NLB. Neither firewalls seemed to be a problem, as the same behavior occurred locally on the server and moreover there was no trace of dropped traffic in the firewall logs system. However looking up particular parts of the error message provided some traces - apparently in some cases error message was caused by improper port redirection, for instance when outgoing port was different from the incoming one. That lead to the configuration of the NLB settings, which turned out to be a culprit
Resolution:
After taking a closer look at the configuration of NLB it turned out, that even though:
1. Traffic IP group was set up correctly and was pointing to a proper NLB address
2. The Pool was configured correctly and contained all the SCOM server nodes inside
3. The Virtual Server was configured to forward port 5724
There was a problem in the setup of Virtual Server. In general this configuration item looks as follows:
Problematic Configuration of NLB Virtual Server for SCOM Console Application
As you can see, even though the forwarded port is properly setup to 5724, the Protocol is incorrectly configured as HTTP. It turned out this was the source of the problem. It was fixed by setting it to Generic Client First, which caused the error message to go away.
Brak komentarzy:
Prześlij komentarz