Server Issues

[Tron]

--------------------------------------------------------------------------------
2021-12-17 22:18:48.025 [Error] Client Packet Error! [Packet: LoginPacket | User: User | Player:  | IP IP]
2021-12-17 22:18:48.025 [Error] 
  Message: Index was outside the bounds of the array.
    Stack:   at (wrapper stelemref) System.Object.virt_stelemref_sealed_class(intptr,object)
  at Intersect.Server.Networking.PacketSender.SendPlayerCharacters (Intersect.Server.Networking.Client client) [0x00177] in <abc18994255647c3bc439dff7e059aeb>:0 
  at Intersect.Server.Networking.PacketHandler.HandlePacket (Intersect.Server.Networking.Client client, Intersect.Network.Packets.Client.LoginPacket packet) [0x00360] in <abc18994255647c3bc439dff7e059aeb>:0 
  at Intersect.Network.PacketHandlerRegistry+<>c__DisplayClass38_1`2[TPacketSender,TPacket].<CreateWeaklyTypedDelegateForMethodInfo>b__1 (Intersect.Network.IPacketSender packetSender, Intersect.Network.IPacket packet) [0x00001] in <0557b77026a34e78af480eff017e5238>:0 
  at Intersect.Server.Networking.PacketHandler.ProcessPacket (Intersect.Network.IPacket packet, Intersect.Server.Networking.Client client) [0x0010e] in <abc18994255647c3bc439dff7e059aeb>:0 
  at Intersect.Server.Networking.Client.HandlePackets () [0x000eb] in <abc18994255647c3bc439dff7e059aeb>:0 


--------------------------------------------------------------------------------

The above is mono-sgen

Below is from playerdbIntersect Server
 

2021-12-17 17:08:04.717 [Error] PLAYERDB: 10000 - An exception occurred in the database while saving changes for context type 'Intersect.Server.Database.PlayerData.PlayerContext'.
Microsoft.EntityFrameworkCore.DbUpdateConcurrencyException: Database operation expected to affect 1 row(s) but actually affected 0 row(s). Data may have been modified or deleted since entities were loaded. See http://go.microsoft.com/fwlink/?LinkId=527962 for information on understanding and handling optimistic concurrency exceptions.
   at Microsoft.EntityFrameworkCore.Update.AffectedCountModificationCommandBatch.ThrowAggregateUpdateConcurrencyException(Int32 commandIndex, Int32 expectedRowsAffected, Int32 rowsAffected)
   at Microsoft.EntityFrameworkCore.Update.AffectedCountModificationCommandBatch.ConsumeResultSetWithoutPropagation(Int32 commandIndex, RelationalDataReader reader)
   at Microsoft.EntityFrameworkCore.Update.AffectedCountModificationCommandBatch.Consume(RelationalDataReader reader)
   at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.Execute(IRelationalConnection connection)
   at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.Execute(DbContext _, ValueTuple`2 parameters)
   at Microsoft.EntityFrameworkCore.Storage.Internal.NoopExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.Execute(IEnumerable`1 commandBatches, IRelationalConnection connection)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(IReadOnlyList`1 entriesToSave)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(Boolean acceptAllChangesOnSuccess)
   at Microsoft.EntityFrameworkCore.DbContext.SaveChanges(Boolean acceptAllChangesOnSuccess)

--------------------------------------------------------------------------------

Any ideas?

[Cheshire]

Lacking some context here, so I'm assuming it's aliens coming to invade our world so they can steal all the sippycups.

 

As for the actual causes? Hard to say, concurrency problems especially are hard to track down since you'll have to figure out what is causing it and there's a lot of things that could go wrong. Although I personally suspect there's some potential for the API to cause this. I'm going to guess this is on a custom build? 

 

Does the first error happen regularly? If so I'd suggest adding the pdb files and compiling in debug mode for line numbers. 

[Tron]

2 hours ago, Cheshire said:

Lacking some context here, so I'm assuming it's aliens coming to invade our world so they can steal all the sippycups.

 

As for the actual causes? Hard to say, concurrency problems especially are hard to track down since you'll have to figure out what is causing it and there's a lot of things that could go wrong. Although I personally suspect there's some potential for the API to cause this. I'm going to guess this is on a custom build? 

 

Does the first error happen regularly? If so I'd suggest adding the pdb files and compiling in debug mode for line numbers. 

 

Yes it's custom and as for regularly I can't be sure. I started having people trickle in to a closed alpha access today.

So far what I've been able to gather is that accounts that were playing at the time of the crash can no longer be accessed. Old accounts can be though so I'm not sure what the issue is yet.

[Cheshire]

To what extent did you add custom changes and which branch was this based off of?

Unfortunately it's pretty hard to guess what's wrong as an outsider so maybe we can figure something out with a little more information.

 

When they log in after causing a concurrency error do they get the first error?

 

Can you confirm their characters still exist and are linked to their accounts?

[Tron]

10 hours ago, Cheshire said:

To what extent did you add custom changes and which branch was this based off of?

Unfortunately it's pretty hard to guess what's wrong as an outsider so maybe we can figure something out with a little more information.

 

When they log in after causing a concurrency error do they get the first error?

 

Can you confirm their characters still exist and are linked to their accounts?

 

It's based on 7.0.0 main branch. There are quite a few custom changes.

 

So basically what seems to happen is after some time the server kinda just seizes up and doesn't allow new connections. Once this happens anyone that is online will no longer be able to log in (except administrator accounts).

I tried checking through the playerdb to see if I could find anything and I can't. 

Below is the error I'm seeing in the console, I removed the user's account name and IP for privacy.

2021-12-18 17:46:48.506 [Error] Client Packet Error! [Packet: LoginPacket | User: <User> | Player:  | IP <IP>]
2021-12-18 17:46:48.508 [Error] {"DataKeys":0,"MessageLength":42,"StackTraceLength":1018,"InnerException":false,"Source":null,"TargetSite":null,"TargetSiteDeclaringType":null,"Type":"System.IndexOutOfRangeException"}
2021-12-18 17:46:48.509 [Error] Client Packet Error! [Packet: LoginPacket | User: <User> | Player:  | IP <IP>]
2021-12-18 17:46:48.509 [Error] See logs for more information.

When this happens to a user they get a connection error that tells them to check their internet connection. Through testing, if I manually deleted a character from an affected account they can then log in again.
Do you think it's possible characters are detaching from their associated account and when they try to log in it shows the account has a character but it can't find it?

[Cheshire]

10 minutes ago, Scaly said:

Do you think it's possible characters are detaching from their associated account and when they try to log in it shows the account has a character but it can't find it?

 

Not naturally, I am suspecting something that's custom is causing exactly that however.

It's a little tricky to tell what exactly is going on not knowing the extent of the changes, however. Though I would definitely look into anything database related.