PDA

View Full Version : Info/Speculation/POV on the Xsyon Server Prelude



l3lue
03-14-2011, 04:50 PM
Proposed Agenda
I decided instead of spamming chat everytime someone ask. I would group all my info into one editable post as I find info out and gather it all here. Feel free to comment, add in your speculation and I will edit it into the main post here complete with credit where credit is due :). I will clean this up over time if it works in a nice neat fashion.

First,Please forgive any typo's, and wrongly typed information. My brain thinks faster than i can comprehend when i'm this tired. Find any errors? Post n leme know please :cool:

I'm with Freedom of information to try and best speculate whats going on with the game cause I feel everyone should know as we all paid for this prospect. At the same time I don't think it is correct to goto Jordi directly demanding his solutions, just not the order of business. In a sense we are virtual shareholders of a virtual company sortof and im not payed or in the board of directors neither is anyone else except maybe staff. My goal here is to provide some of my limited technical knowledge to try and feed whats going on to the shareholders in the most accurate and informative way that I can. While not going over people's heads that don't live and breath computers.

People speculating branches into tech savy people, the people that "think" they know how the server rolls, and angry people who want something to complain about...that is also only speculation ;) My job I will take on is to filter all results here from what i read and find also what people post here.

Current State: Unknown, Speculation Stage

Current Discussed Issues

Possible Code Bug: Long tribe names?
Data Corruption
CPU Stress
Hardware
Weather System: Rain


Possible Server Packages




Single (1x) Core

Processor Hard Disk Memory Bandwidth
1x Intel Celeron D 325 - 2.53 GHz 160 GB SATAII 1 GB DDR2 6,000 GB
1x Intel Celeron D 325 - 2.53 GHz 250 GB SATAII 2 GB DDR2 6,000 GB

Quad (4x) Core

Processor Hard Disk Memory Bandwidth
1x Intel Xeon Lynnfield x3430 - 2.4 GHz 250 GB SATAII 2 GB DDR3 ECC 6,000 GB
1x Intel Xeon Lynnfield x3460 - 2.8 GHz 500 GB SATAII 4 GB DDR3 ECC 6,000 GB
1x Intel Xeon Nehalem e5520 - 2.26 GHz 500 GB SATAII 6 GB DDR3 ECC 6,000 GB

Eight (8x) Core

Processor Hard Disk Memory Bandwidth
2x Intel Xeon Nehalem e5520 - 2.26GHz 500 GB SATAII 12 GB DDR3 ECC 6,000 GB
2x Intel Xeon Nehalem e5520 - 2.26GHz 2 x 500 GB SATAII - RAID 1 12 GB DDR3 ECC 6,000 GB






Best one I found, others wouldn't work I don't think

Hardware Specifications

Dual CPU
Brand: Intel
Model: Xeon L5520
Speed: 2.26Ghz
Core: Quad Core
Total: 8 CPU Cores
Ram: 12GB
Harddrive: 250GB
Port Speed: 100 Mbps
Bandwidth: 5000GB Tier 1
IP's: 5
200+PLAYER RANGE

My starting Speculation topics, subject to change.



Latency, Bandwidth


Although the theoretical peak bandwidth of a network connection is fixed according to the technology used, the actual bandwidth you will obtain varies over time and is affected by high latencies. Excessive latency creates bottlenecks that prevent data from filling the network pipe, thus decreasing effective bandwidth. The impact of latency on network bandwidth can be temporary (lasting a few seconds) or persistent (constant) depending on the source of the delays.

Sounds interesting right? We are all getting delays and that stuff sounds good together. Cept if the ping rate is correct ingame as I see it when we all get the lag, i shift between 31ms which is instant then jumping from 100~1000 with as much as 8-15min delays between game actions. Not only are the ping levels slightly above average (I played WoW on a eastern server and got 100-400ms delays and everything was playable) but the math does not add up vs the time delay. If 31 for me is instant, and ive heard 60 or so is also average with instant results from others, why does it only goto 400-900 before it takes 15 mins between actions? Just does not fit IMO.

CPU

CPU load high = low/average ping, horrible performance.... layman example:

McDonalds best employee is by himself with noone else at 12PM, Load is high, Your sight to the building is fast, your smell of the burgers is instant...but the load on the 1 employee means instead of multiple people working to fix your food, you got 1. You have a worker latency issue and before he takes the money, processes the order, cooks the food, gives it to you, can expect to wait quite awhile for the people infront of you.

I know there is QoS type option for CPU which is labeled Priority in Windows, (may be different in linux, and I HOPE the server is linux), but how will that fix the problem here? There is people ingame and its correct to process task in the game based on the order received logically. So that's out.

Bigger better box, superfast multicore processor, loads of ram, multi threading built into the Xsyon host software if (if not already)...possibly could fix the issue, multi threading is a nightmare....at least for my poor attempts at it.

Code

Jordi knows, we don't unless he tells us...nuff said
Code rely's on the coder(s) writing it. There is little speculation to that problem obviously...All i will say is this, The McDonalds employees serving out the food is directed by the main boss, and depending on his code orders, can be faster, or slower, code optimization in a layman nutshell :)

Ending, Non-constructive short story

I have worked with servers for a electric corporation, I did not do the code, but I helped track down the source of the tx/rx halt/severe delay AKA "Lag" in the customer accounting software. Yea nothing impressive.

The customer base was over 800,000+ and there was about 2,000+ employees across the state with access to it all interlinked and connected through their intranet via VPN. Poof it dies suddenly...

I seen high dollar certificated experts come in, stand around with their hands on their chin, about 500 bux an hr. Then a indian programmer with a even more expensive price tag and extreeeeemly rare and hard to find people with the exclusive code knowledge of the specific guarded syntax that coded the software....i think only 8 in the world (also controls the hydroelectric system completely)

On top of that the COO was jumping the admin's case, and he was jumping our case ect.

After tearing apart the whole network of the central hub, digging through all the Ethernet cable in the switching room, 1 cable led back into the same switch....fixed it, no more issue, problem solved.

Problems aren't always simple but with simple patience it will all be sorted out eventually. Some people work on it, others speculate, some people rant and rage, others throw money at the problem to try and make it disappear faster.

Saolite
03-14-2011, 05:39 PM
In all reality, the ping displayed itself is ambiguous -- is it ever stated that it is the current ping, or is it an overall average ping across X seconds?

Out of personal belief of how the system has been behaving thus far, the ping that we see in Xsyon is the average ping across X amount of seconds -- so when it spikes, you're not actually getting a sudden "record high". Your actually getting significantly more than the displayed amount. But the reverse is also true -- when we lag, we sometimes see our ping suddenly lower than our average ping when we're not lagging. Which, if we're lagging consistently, doesn't make sense to me. It means, however, that there is likely a bottleneck somewhere; code that's off, hardware that's not performing adequately, etc. Something, somewhere, is is causing data to not filter through at a consistent rate, which causes the low-spikes and high-spikes in lag.

CPU

As far as hardware is concerned, it could be any number of things in that area that can cause lag. A single loose cable on a modem at a house network can, for example, cause intermittent modem resets and extreme signal loss that makes it seem like "lag" but is actually complete halt of service for X amount of seconds / minutes.

When dealing with server equipment, it gets a little more complicated. Switches need to be addressed, ports need to be addressed on the software side, then set how much bandwidth each port is allocated ( if micromanaged to that level ). I haven't used Linux for a server enviroment, but I have Windows Server 2003 R2 and Windows Server 2008 R2; both are very ' simple ' to configure and manage, but of course suffer from server issues and are very unlike a typical Windows operating system in terms of what it does. Shutting them down makes them go crazy, basically. Servers are not meant to be shut down. And then there's whole hosts of problems on that end that can rise up, as well; again, ports need to be opened, bandwidth allocated, services enables, permissions, restrictions, etc.

Though what physically their server is capable of? I really have no idea. But it's a good thing they're not using the patch server as the game server -- that's a step in the right direction. I've worked on blade servers, rack servers, and home-hosted Team Fortress 2 desktop servers. They all have their different strengths to them, so I couldn't even begin to guess at what kind of hardware they're running or how many they've got linked together for performance.

More hardware / better hardware is never a bad thing from a corporate standpoint. Why? It means you have some sort of redundancy once you do go over what you actually need, and with growth you'll have the hardware already ready to meet the demands of that growth. And you would need some pretty beefy servers to run Xsyon decently.


On the non-hardware side, it's about coding, like you mentioned, which we know nothing about except what we're told. Also, though, you have to take into account that they're also limited by the amount of bandwidth they can use through their ISP. I imagine they have business accounts and what not, but ISP's generally tend to only guarantee up to a certain amount, and after that it's "as its available".

tldr; a lot can go wrong. One little thing that's off can throw an entire system into chaos and finding the fix isn't as simple as bandwidth testing every individual component.

JCatano
03-14-2011, 05:44 PM
Though what physically their server is capable of? I really have no idea. But it's a good thing they're not using the patch server as the game server -- that's a step in the right direction. I've worked on blade servers, rack servers, and home-hosted Team Fortress 2 desktop servers. They all have their different strengths to them, so I couldn't even begin to guess at what kind of hardware they're running or how many they've got linked together for performance.

More hardware / better hardware is never a bad thing from a corporate standpoint. Why? It means you have some sort of redundancy once you do go over what you actually need, and with growth you'll have the hardware already ready to meet the demands of that growth. And you would need some pretty beefy servers to run Xsyon decently.

My post from a past thread -

Looks to be one of these per WhoIs:

http://www.ubiquityservers.com

Or...

http://www.hypernia.com


Neither looks to be anything special.

ifireallymust
03-14-2011, 06:04 PM
Deleted, just in case I'm closer to being right than I think I am. -.-

jumpshot
03-14-2011, 06:29 PM
someone paraphrase this for me

rajaxone
03-14-2011, 07:05 PM
someone paraphrase this for me

Basically, server lag can be caused by programming, server infrastructural planning or very simple things like bad cables or connections. :)

jumpshot
03-14-2011, 07:10 PM
thanks Ghazi!

Sorakin
03-14-2011, 07:25 PM
Well currently it looks like a data corruption issue causing automated rollbacks so no bad data is saved. I don't know if anything changed earlier from the lag people were experiencing, but I never got any lag after I started playing tonight (around 9PM EST). Data corruption is most assuredly a code issue, or bad HW issue... but more than likely a code issue. There were no data corruption issues for MOST of the last week until people started hacking up the character name checker.

Given that the only previously known issue of data corruption was caused by players with longer than expected (or allowed) names, have the devs made sure no issue exists within the tribe naming process?

Also as we all know most of the actions were disabled this past week, these could be suspect, but they have been enabled in beta before without issue.

Banano
03-14-2011, 08:13 PM
Why does it always rain when the server crashes :/ ?

ifireallymust
03-14-2011, 08:15 PM
Well currently it looks like a data corruption issue causing automated rollbacks so no bad data is saved. I don't know if anything changed earlier from the lag people were experiencing, but I never got any lag after I started playing tonight (around 9PM EST). Data corruption is most assuredly a code issue, or bad HW issue... but more than likely a code issue. There were no data corruption issues for MOST of the last week until people started hacking up the character name checker.

Given that the only previously known issue of data corruption was caused by players with longer than expected (or allowed) names, have the devs made sure no issue exists within the tribe naming process?

Also as we all know most of the actions were disabled this past week, these could be suspect, but they have been enabled in beta before without issue.

Errr....

*looks at two word tribe name*

Solarwest
03-14-2011, 08:17 PM
Why does it always rain when the server crashes :/ ?

Blame it on the Rain!! :D

Banano
03-14-2011, 08:20 PM
Given that the only previously known issue of data corruption was caused by players with longer than expected (or allowed) names, have the devs made sure no issue exists within the tribe naming process?

Quite possible.


Blame it on the Rain!! :D

It's just a reoccuring thing I've been noticing :s.

l3lue
03-14-2011, 08:23 PM
In all reality, the ping displayed itself is ambiguous -- is it ever stated that it is the current ping, or is it an overall average ping across X seconds?

Out of personal belief of how the system has been behaving thus far, the ping that we see in Xsyon is the average ping across X amount of seconds -- so when it spikes, you're not actually getting a sudden "record high". Your actually getting significantly more than the displayed amount. But the reverse is also true -- when we lag, we sometimes see our ping suddenly lower than our average ping when we're not lagging. Which, if we're lagging consistently, doesn't make sense to me. It means, however, that there is likely a bottleneck somewhere; code that's off, hardware that's not performing adequately, etc. Something, somewhere, is is causing data to not filter through at a consistent rate, which causes the low-spikes and high-spikes in lag.
Yes! I like your explanation. Thanks so much for taking the time to help contribute! I'm brainstorming how to merge everyones data while also giving credit instead of a stream of quotes on the main post. Think I will have a credits section and generalize it all maybe.


CPU

As far as hardware is concerned, it could be any number of things in that area that can cause lag. A single loose cable on a modem at a house network can, for example, cause intermittent modem resets and extreme signal loss that makes it seem like "lag" but is actually complete halt of service for X amount of seconds / minutes.

When dealing with server equipment, it gets a little more complicated. Switches need to be addressed, ports need to be addressed on the software side, then set how much bandwidth each port is allocated ( if micromanaged to that level ). I haven't used Linux for a server enviroment, but I have Windows Server 2003 R2 and Windows Server 2008 R2; both are very ' simple ' to configure and manage, but of course suffer from server issues and are very unlike a typical Windows operating system in terms of what it does. Shutting them down makes them go crazy, basically. Servers are not meant to be shut down. And then there's whole hosts of problems on that end that can rise up, as well; again, ports need to be opened, bandwidth allocated, services enables, permissions, restrictions, etc.

Though what physically their server is capable of? I really have no idea. But it's a good thing they're not using the patch server as the game server -- that's a step in the right direction. I've worked on blade servers, rack servers, and home-hosted Team Fortress 2 desktop servers. They all have their different strengths to them, so I couldn't even begin to guess at what kind of hardware they're running or how many they've got linked together for performance.

More hardware / better hardware is never a bad thing from a corporate standpoint. Why? It means you have some sort of redundancy once you do go over what you actually need, and with growth you'll have the hardware already ready to meet the demands of that growth. And you would need some pretty beefy servers to run Xsyon decently.


On the non-hardware side, it's about coding, like you mentioned, which we know nothing about except what we're told. Also, though, you have to take into account that they're also limited by the amount of bandwidth they can use through their ISP. I imagine they have business accounts and what not, but ISP's generally tend to only guarantee up to a certain amount, and after that it's "as its available".

tldr; a lot can go wrong. One little thing that's off can throw an entire system into chaos and finding the fix isn't as simple as bandwidth testing every individual component.

Again NICE input here. A fiber line for the branch I worked for was around $16,000 per month...but it was spliced directly into the backbone, resell rights (they had their own internet for power customers), Business lines CAN get expensive and notelling what the box he has is limited to. One thing I know ISP's can shaft you on is upload. I can get a nice download speed cheap, When you start requesting upload higher then it goes into a higher "business" tier. My own home to get a upload speed of 1.5mb and download of 1.5mb, local ISP wants $199.95 per month...or the "Business" package as they call it....how about 3mb up/down? $349..

There are exceptions. Fiber access supply/demand. Iowa you can get somewhere around 50-80gb down AND up for $60 WITH a IP Phone and TV package included...makes me wanna die...or move to Iowa :mad: anyways back to subject


My post from a past thread -

Looks to be one of these per WhoIs:

http://www.ubiquityservers.com

Or...

http://www.hypernia.com


Neither looks to be anything special.

GREAT! I remember that post but I forgot while I made the topic, I will check there packages and get details on here. Thanks much for the contribution :cool:



Deleted, just in case I'm closer to being right than I think I am. -.-

bahhh, I didnt get to read what you put :( It dont matter if your right or wrong or in your case close to being right. I've read alot of post from you here and like your input as well :). You had some nice comments on junkpiles, social structure, and crowding of areas and also Safe Zones at Totems...I think, but yeh, speak your mind.


Basically, server lag can be caused by programming, server infrastructural planning or very simple things like bad cables or connections.

yap, thanks for paraphrasing bud :)


Well currently it looks like a data corruption issue causing automated rollbacks so no bad data is saved. I don't know if anything changed earlier from the lag people were experiencing, but I never got any lag after I started playing tonight (around 9PM EST). Data corruption is most assuredly a code issue, or bad HW issue... but more than likely a code issue. There were no data corruption issues for MOST of the last week until people started hacking up the character name checker.

Given that the only previously known issue of data corruption was caused by players with longer than expected (or allowed) names, have the devs made sure no issue exists within the tribe naming process?

Also as we all know most of the actions were disabled this past week, these could be suspect, but they have been enabled in beta before without issue.

Thats good, knowledge of the save and self restart system. I do agree while it is annoying it IS nice to have non-corrupt saves as well as giving everyone the ability to roll back and do what they did different/again for testing to see if they crash the server. I remember 2 days into playing I thought i crashed the server cause i got to the highest mountain top and was jumping/spinning to stress test my PC and the server glitched out, water in the lake far below messed up, crashed and i was back at the bottom of the mountain upon the server restart. I went and tested again, no crash :)


Why does it always rain when the server crashes :/ ?

I noticed that as well. I was even quick to point to the weather system and how it migrates, clouds form ect. I seen no patterns though with that when ive been on. I seen it twice in a row, then 3rd time fine...and this was after we did the final wipe il add, it rained bunches before the wipe, no issues... Thanks for the contribution :cool:


Errr....

*looks at two word tribe name*

lol, contribution to a smile :) Thanks :cool:


Thanks again to everyone taking the time to contribute. I think if we get this built well enough maybe it will get Jordi's attention or a Guide/mod to give an official statement.

Sorry for such a long reply but I aim to thank, work in, everyones data at some point. I would like to make a wikia or something where everyone can contribute and change it how they like...but not so sure how everyone would feel about that extra hoop to jump through..prolly overkill heh

I thought about seeing if I can get a phonecall/txt/email or access to a dev log to see if I can help with the information transfer from Dev <> customer. I do agree the best thing to do sometimes is not say anything at all but egh...so many disheartened people ive talked to just wanna explanation and they could be happy, and are close to giving up hope if its the same launch day.

Solarwest
03-14-2011, 08:24 PM
Quite possible.



It's just a reoccuring thing I've been noticing :s.

I agree, has to do something with the rain coming on. Just my lame old self thinking of the Milli Vanilli song "Blame it on the Rain"

Sorakin
03-14-2011, 08:37 PM
I think this is the first possibly productive thread I've seen on these boards. Kudos!

Added after 9 minutes:

I just remembered that quests could also be abused by that name thing... really anything that you enter in text for :)

l3lue
03-14-2011, 08:54 PM
Anyone else notice the on and off snow patches on the ground? Im thinking there may been work on the weather system before the wipe. It was working before, it was springtime or summer. They may have did work on the system in the meantime and since this is the first time back to winter maybe having some issues with Rain supposed to change to snow.

To contradict myself it could be the other way around. The little bit of snow comes and the rain comes and makes it go away, and that causes an issue or bug in the code. Need to get more testing done I suppose. But literaly ive seen both it appear during server start, and disapear sometime later afterwards

All i know is snow patches come and go and the weather patterns seem to be repeating and server crashes sometimes during the start of a rain...

Delvie
03-14-2011, 09:15 PM
Hardware has been mentioned and code has been mentioned but no one has mentioned database yet. This is separate from the code depending on how many checks and balances there are on the database itself. Not to mention that optimizing db searches is an art form.

wolfmoonstrike
03-14-2011, 09:55 PM
Anyone else notice the on and off snow patches on the ground? Im thinking there may been work on the weather system before the wipe. It was working before, it was springtime or summer. They may have did work on the system in the meantime and since this is the first time back to winter maybe having some issues with Rain supposed to change to snow.

To contradict myself it could be the other way around. The little bit of snow comes and the rain comes and makes it go away, and that causes an issue or bug in the code. Need to get more testing done I suppose. But literaly ive seen both it appear during server start, and disapear sometime later afterwards

All i know is snow patches come and go and the weather patterns seem to be repeating and server crashes sometimes during the start of a rain...


It isn't snow....(fooled us too) The "snow patches" are actually rain puddles lol.

l0c0ninja
03-14-2011, 10:11 PM
Its gods will. We dont need to worry everything is for a reason. God doesnt want us to play right now so he will force his will. You can send donations to the church through my paypal if you want to see more Miracles

sudo
03-14-2011, 10:31 PM
Erm...
We aren't shareholders, we are costumers. We are never gonna profit from this product in any material way (not legally anyway).
We are nowhere near a part of the company but are only buying its product. You could say that all of us are donors of some kind, since we all knew that we paid for an unfinished product and are even ready to keep paying for it for the next half year-year while the developers are working on finishing it.
Still, the difference between a costumer and a shareholder is quite different. Sadly, I wouldn't want to be a Xsyon shareholder at the moment...

BigCountry
03-15-2011, 05:26 AM
Man, my development box at work has more ram than a few of his boxes....
:(

I do not think it's hardware or bandwidth though. You can fix those both with money, not brain power, that's an easy fix.

It's his code. I wish I knew more about it so we could help. But he has created a custom engine, so there is no telling where the bottleneck could be without looking at his architecture and execution.
:(