Info/Speculation/POV on the Xsyon Server Prelude

[l3lue]

Proposed Agenda
I decided instead of spamming chat everytime someone ask. I would group all my info into one editable post as I find info out and gather it all here. Feel free to comment, add in your speculation and I will edit it into the main post here complete with credit where credit is due . I will clean this up over time if it works in a nice neat fashion.

First,Please forgive any typo's, and wrongly typed information. My brain thinks faster than i can comprehend when i'm this tired. Find any errors? Post n leme know please

I'm with Freedom of information to try and best speculate whats going on with the game cause I feel everyone should know as we all paid for this prospect. At the same time I don't think it is correct to goto Jordi directly demanding his solutions, just not the order of business. In a sense we are virtual shareholders of a virtual company sortof and im not payed or in the board of directors neither is anyone else except maybe staff. My goal here is to provide some of my limited technical knowledge to try and feed whats going on to the shareholders in the most accurate and informative way that I can. While not going over people's heads that don't live and breath computers.

People speculating branches into tech savy people, the people that "think" they know how the server rolls, and angry people who want something to complain about...that is also only speculation My job I will take on is to filter all results here from what i read and find also what people post here.

Current State: Unknown, Speculation Stage

Current Discussed Issues

1. Possible Code Bug: Long tribe names?
2. Data Corruption
3. CPU Stress
4. Hardware
5. Weather System: Rain

Possible Server Packages

Single (1x) Core

Processor	                          Hard Disk	    Memory	       Bandwidth
1x Intel Celeron D 325 - 2.53 GHz	160 GB SATAII	1 GB DDR2	6,000 GB		
1x Intel Celeron D 325 - 2.53 GHz	250 GB SATAII	2 GB DDR2	6,000 GB		

Quad (4x) Core

Processor	                            Hard Disk	    Memory            	Bandwidth
1x Intel Xeon Lynnfield x3430 - 2.4 GHz	250 GB SATAII	2 GB DDR3 ECC	6,000 GB	
1x Intel Xeon Lynnfield x3460 - 2.8 GHz	500 GB SATAII	4 GB DDR3 ECC	6,000 GB	
1x Intel Xeon Nehalem e5520 - 2.26 GHz	500 GB SATAII	6 GB DDR3 ECC	6,000 GB	

Eight (8x) Core

Processor	                          Hard Disk	          Memory	          Bandwidth	
2x Intel Xeon Nehalem e5520 - 2.26GHz	500 GB SATAII	12 GB DDR3 ECC	6,000 GB	
2x Intel Xeon Nehalem e5520 - 2.26GHz	2 x 500 GB SATAII - RAID 1	12 GB DDR3 ECC	6,000 GB

Best one I found, others wouldn't work I don't think

Hardware Specifications

Dual CPU
Brand: Intel
Model: Xeon L5520
Speed: 2.26Ghz
Core: Quad Core
Total:  8 CPU Cores
Ram: 12GB
Harddrive: 250GB
Port Speed: 100 Mbps
Bandwidth: 5000GB Tier 1
IP's: 5
200+PLAYER RANGE

My starting Speculation topics, subject to change.



Latency, Bandwidth

Although the theoretical peak bandwidth of a network connection is fixed according to the technology used, the actual bandwidth you will obtain varies over time and is affected by high latencies. Excessive latency creates bottlenecks that prevent data from filling the network pipe, thus decreasing effective bandwidth. The impact of latency on network bandwidth can be temporary (lasting a few seconds) or persistent (constant) depending on the source of the delays.

Sounds interesting right? We are all getting delays and that stuff sounds good together. Cept if the ping rate is correct ingame as I see it when we all get the lag, i shift between 31ms which is instant then jumping from 100~1000 with as much as 8-15min delays between game actions. Not only are the ping levels slightly above average (I played WoW on a eastern server and got 100-400ms delays and everything was playable) but the math does not add up vs the time delay. If 31 for me is instant, and ive heard 60 or so is also average with instant results from others, why does it only goto 400-900 before it takes 15 mins between actions? Just does not fit IMO.

CPU

CPU load high = low/average ping, horrible performance.... layman example:

McDonalds best employee is by himself with noone else at 12PM, Load is high, Your sight to the building is fast, your smell of the burgers is instant...but the load on the 1 employee means instead of multiple people working to fix your food, you got 1. You have a worker latency issue and before he takes the money, processes the order, cooks the food, gives it to you, can expect to wait quite awhile for the people infront of you.

I know there is QoS type option for CPU which is labeled Priority in Windows, (may be different in linux, and I HOPE the server is linux), but how will that fix the problem here? There is people ingame and its correct to process task in the game based on the order received logically. So that's out.

Bigger better box, superfast multicore processor, loads of ram, multi threading built into the Xsyon host software if (if not already)...possibly could fix the issue, multi threading is a nightmare....at least for my poor attempts at it.

Code

Jordi knows, we don't unless he tells us...nuff said
Code rely's on the coder(s) writing it. There is little speculation to that problem obviously...All i will say is this, The McDonalds employees serving out the food is directed by the main boss, and depending on his code orders, can be faster, or slower, code optimization in a layman nutshell

Ending, Non-constructive short story

I have worked with servers for a electric corporation, I did not do the code, but I helped track down the source of the tx/rx halt/severe delay AKA "Lag" in the customer accounting software. Yea nothing impressive.

The customer base was over 800,000+ and there was about 2,000+ employees across the state with access to it all interlinked and connected through their intranet via VPN. Poof it dies suddenly...

I seen high dollar certificated experts come in, stand around with their hands on their chin, about 500 bux an hr. Then a indian programmer with a even more expensive price tag and extreeeeemly rare and hard to find people with the exclusive code knowledge of the specific guarded syntax that coded the software....i think only 8 in the world (also controls the hydroelectric system completely)

On top of that the COO was jumping the admin's case, and he was jumping our case ect.

After tearing apart the whole network of the central hub, digging through all the Ethernet cable in the switching room, 1 cable led back into the same switch....fixed it, no more issue, problem solved.

Problems aren't always simple but with simple patience it will all be sorted out eventually. Some people work on it, others speculate, some people rant and rage, others throw money at the problem to try and make it disappear faster.

[Saolite]

In all reality, the ping displayed itself is ambiguous -- is it ever stated that it is the current ping, or is it an overall average ping across X seconds?

Out of personal belief of how the system has been behaving thus far, the ping that we see in Xsyon is the average ping across X amount of seconds -- so when it spikes, you're not actually getting a sudden "record high". Your actually getting significantly more than the displayed amount. But the reverse is also true -- when we lag, we sometimes see our ping suddenly lower than our average ping when we're not lagging. Which, if we're lagging consistently, doesn't make sense to me. It means, however, that there is likely a bottleneck somewhere; code that's off, hardware that's not performing adequately, etc. Something, somewhere, is is causing data to not filter through at a consistent rate, which causes the low-spikes and high-spikes in lag.

CPU

As far as hardware is concerned, it could be any number of things in that area that can cause lag. A single loose cable on a modem at a house network can, for example, cause intermittent modem resets and extreme signal loss that makes it seem like "lag" but is actually complete halt of service for X amount of seconds / minutes.

When dealing with server equipment, it gets a little more complicated. Switches need to be addressed, ports need to be addressed on the software side, then set how much bandwidth each port is allocated ( if micromanaged to that level ). I haven't used Linux for a server enviroment, but I have Windows Server 2003 R2 and Windows Server 2008 R2; both are very ' simple ' to configure and manage, but of course suffer from server issues and are very unlike a typical Windows operating system in terms of what it does. Shutting them down makes them go crazy, basically. Servers are not meant to be shut down. And then there's whole hosts of problems on that end that can rise up, as well; again, ports need to be opened, bandwidth allocated, services enables, permissions, restrictions, etc.

Though what physically their server is capable of? I really have no idea. But it's a good thing they're not using the patch server as the game server -- that's a step in the right direction. I've worked on blade servers, rack servers, and home-hosted Team Fortress 2 desktop servers. They all have their different strengths to them, so I couldn't even begin to guess at what kind of hardware they're running or how many they've got linked together for performance.

More hardware / better hardware is never a bad thing from a corporate standpoint. Why? It means you have some sort of redundancy once you do go over what you actually need, and with growth you'll have the hardware already ready to meet the demands of that growth. And you would need some pretty beefy servers to run Xsyon decently.


On the non-hardware side, it's about coding, like you mentioned, which we know nothing about except what we're told. Also, though, you have to take into account that they're also limited by the amount of bandwidth they can use through their ISP. I imagine they have business accounts and what not, but ISP's generally tend to only guarantee up to a certain amount, and after that it's "as its available".

tldr; a lot can go wrong. One little thing that's off can throw an entire system into chaos and finding the fix isn't as simple as bandwidth testing every individual component.

[JCatano]

Though what physically their server is capable of? I really have no idea. But it's a good thing they're not using the patch server as the game server -- that's a step in the right direction. I've worked on blade servers, rack servers, and home-hosted Team Fortress 2 desktop servers. They all have their different strengths to them, so I couldn't even begin to guess at what kind of hardware they're running or how many they've got linked together for performance.

More hardware / better hardware is never a bad thing from a corporate standpoint. Why? It means you have some sort of redundancy once you do go over what you actually need, and with growth you'll have the hardware already ready to meet the demands of that growth. And you would need some pretty beefy servers to run Xsyon decently.

My post from a past thread -

Looks to be one of these per WhoIs:

http://www.ubiquityservers.com

Or...

http://www.hypernia.com


Neither looks to be anything special.

[ifireallymust]

Deleted, just in case I'm closer to being right than I think I am. -.-

[jumpshot]

someone paraphrase this for me

[rajaxone]

someone paraphrase this for me

Basically, server lag can be caused by programming, server infrastructural planning or very simple things like bad cables or connections.

[jumpshot]

thanks Ghazi!

[Sorakin]

Well currently it looks like a data corruption issue causing automated rollbacks so no bad data is saved. I don't know if anything changed earlier from the lag people were experiencing, but I never got any lag after I started playing tonight (around 9PM EST). Data corruption is most assuredly a code issue, or bad HW issue... but more than likely a code issue. There were no data corruption issues for MOST of the last week until people started hacking up the character name checker.

Given that the only previously known issue of data corruption was caused by players with longer than expected (or allowed) names, have the devs made sure no issue exists within the tribe naming process?

Also as we all know most of the actions were disabled this past week, these could be suspect, but they have been enabled in beta before without issue.

[Banano]

Why does it always rain when the server crashes :/ ?

[ifireallymust]

Well currently it looks like a data corruption issue causing automated rollbacks so no bad data is saved. I don't know if anything changed earlier from the lag people were experiencing, but I never got any lag after I started playing tonight (around 9PM EST). Data corruption is most assuredly a code issue, or bad HW issue... but more than likely a code issue. There were no data corruption issues for MOST of the last week until people started hacking up the character name checker.

Given that the only previously known issue of data corruption was caused by players with longer than expected (or allowed) names, have the devs made sure no issue exists within the tribe naming process?

Also as we all know most of the actions were disabled this past week, these could be suspect, but they have been enabled in beta before without issue.

Errr....

*looks at two word tribe name*