Reply to: System architecture opinions sought

[CheeseSlice]

http://blogs.msdn.com/clustering/arc...4/8648702.aspx

HTH

Yep, uses MS SCUM 2007... don't get me started on that one

[minestrone]

I don't that a multithreaded garbage collected environment could be classed as real-time.

[DimPrawn]

I wouldn't touch Windows NLB with a barge pole if this is for a highly-available production service. App health awareness is non-existent. I've seen NLB systems loadbalancing requests quite happily to systems throwing .NET server errors. There are workarounds, but not simple. With the hardware option you can at least put some intelligence in there to spot this.

Could talk for hours about this subject, but I charge for that

http://blogs.msdn.com/clustering/arc...4/8648702.aspx

HTH

[BrilloPad]

Says who?

By that definition, Royal Mail 1st Class is realtime.

Is it quantifiable? It is supposed to take a day but can be 3 weeks in out part of LOndon.

[DimPrawn]

Realtime means that the time taken is quantifiable, not that it's fast.

Says who?

By that definition, Royal Mail 1st Class is realtime.

[CheeseSlice]

Any thoughts on load balancing hardware? I'm aware of the Barracuda 640 that is able to handle 2000tps and redundant configuration for high availability http://www.barracudanetworks.com/ns/...cer_models.php but I would be interested to learn about other possible options.

Not used one of these, but have used Brocade ServerIrons before (was Foundry before Brocade purchased them).
These are robust pieces of kit and used by the likes of telcos.
They'll do lots of different LB algorithms, round-robin, least loaded, sticky, non-sticky, etc, etc.
Depends what your requirements are and how much you're willing to spend.

I wouldn't touch Windows NLB with a barge pole if this is for a highly-available production service. App health awareness is non-existent. I've seen NLB systems loadbalancing requests quite happily to systems throwing .NET server errors. There are workarounds, but not simple. With the hardware option you can at least put some intelligence in there to spot this.

Could talk for hours about this subject, but I charge for that

[Churchill]

Realtime means that the time taken is quantifiable, not that it's fast.

Stick to noddy .NET "apps".

[TheRefactornator]

UDP is used by all popular online games.

It is indeed not reliable protocol but you can easily solve this by adding counters and confirmations via UDP too.

UDP is ideal for small packets and cases when server needs to process lots of those packets.

Are you going to run this on LAN? If not you need to think about security for the data you send over public networks.

I stand corrected. If I understand you correctly when you mention counters and confirmations, what you are describing here is a application level request/response protocol built on top of UDP that manages network retries to guarrantee delivery. Is that what you were thinking?

This will not run on the LAN so I've already considered encryption, but the sensitivity of the data will need to be confirmed by the vendor so I'm not sure if encryption is required yet.

[AtW]

I did consider this as an option but as I understand it, UDP does not transport well over the internet because UDP packets are likely to be dropped by routers along the path.

UDP is used by all popular online games.

It is indeed not reliable protocol but you can easily solve this by adding counters and confirmations via UDP too.

UDP is ideal for small packets and cases when server needs to process lots of those packets.

Are you going to run this on LAN? If not you need to think about security for the data you send over public networks.

[TheRefactornator]

Using UDP rather than TCP in this case might make sense.

I did consider this as an option but as I understand it, UDP does not transport well over the internet because UDP packets are likely to be dropped by routers along the path. The network topology in this application must use the internet because the data gatherers are at geographically worldwide remote locations relative to the core server's location. I'll be happily corrected if I'm wrong about UDP however.

[AtW]

Using UDP rather than TCP in this case might make sense.

[TheRefactornator]

With regards to the data, what happens if data is lost, do you need guaranteed delivery of data to the server? Retries? Is the order the data is received important?

Would bog std Windows Server NLB give you the scalability you require?

It is not too critical if data is lost occasionally because the data gathering nodes will be running a loop forever gathering data from remote field devices over RS232/485 network, but it will only be only delta change that a transaction will be created for transport to the core. Obviously data loss should be minimal but if it happens it won't be a disaster..order of data received is not important at all.

The serial comms layer will implement it's own application level retry mechanism, but to keep TCP/IP network traffic to a minimum the socket layer is unlikely to be implemented in a request/response type arrangement. It's more likely to simply be one where the data gatherer packages up a handful of transactions, sends and forgets until next time.

I really have no idea about Windows Server NLB and it's capabilities but thanks for the pointer - it is something to consider.

[AtW]

In fact the system being discussed here is a large distributed model of an existing smaller localized system where UI latency actually is 1 second. 1 second has always been perceived by the users and the system vendor to be real-time in this application.

So looks like I got the situation right (term "real-time" is different in length depending on context) and our local dawggy got it wrong...

[TheRefactornator]

Actual length of "real-time" depends on a particular situation, in this case 1 second should be pretty real time (0.2 sec updates are not real time in embedded sense)

In fact the system being discussed here is a large distributed model of an existing smaller localized system where UI latency actually is 1 second. 1 second has always been perceived by the users and the system vendor to be real-time in this application.

[DimPrawn]

With regards to the data, what happens if data is lost, do you need guaranteed delivery of data to the server? Retries? Is the order the data is received important?

Would bog std Windows Server NLB give you the scalability you require?