distributed computing - Client per MicroService vs Generic Client | Who is responsible for microservice client?

I have a microService architecture with 10 microServices and each of those provides a client. Inside of that client which is managed/controlled by microService team we just receive the parameters and pass them to a generic http invoker which receives the endpoint and N params and then does the call.All microService use http and web api (I guess technology doesn't matter).For me doesn't make sense to be the microService team to provide a client, should be the responsibility of the consumer, if they want to create some abstractions or invoke it d...Read more

distributed computing - What are the differences between IPFS and hyperdrive?

A year ago I attended Munich meetup where I first saw hyperdrive project live, and found it impressive. I also recently stumpled upon IPFS –the InterPlanetary file system– and I wonder about the differences. It's easy to have a lot of repercusion when starting a project like this in California, so I guess they might be similar projects but still don't understand the scope of hyperdrive fully.I very much like the API and implementation hyperdrive, so if I had to build an app over a distributed file system I believe it would be easier for me at t...Read more

distributed computing - Committee change in PBFT

I'm implementing a distributed system using Practical Byzantine Fault Tolerance. This method entrusts a committee to vote for each commit. However, if they are all crashed or under DDoS attack the entire network shall breakdown. I'm curious if there's a way to change the entire committee. This is not the same as a view change because view change only replaces faulty nodes, while this requires all committee members to be replaced, thus forming a dynamic committee....Read more

How does "distributed computing" apply to web development or programming in general?

I am about to use Apache Hadoop, the headlines read: The Apache Hadoop project develops open-source software for reliable, scalable, distributed computing.I can relate "scalability" to programming, but I just don't know how this "distributing" can help me in my development. According to wikipedia: A distributed system consists of multiple autonomous computers that communicate through a computer network. The computers interact with each other in order to achieve a common goalSo does this mean I can deploy my web apps across multiple com...Read more

Partial ordering of events in a distributed system

I was wondering if someone could explain in layman's terms what partial ordering of events are in a distributed system? Also, what is total ordering?I would really appreciate this. I've looked all over the web and all I can find are mathematical equations defining partial and total ordering, but not in the context of a distributed system.Thanks very much...Read more

distributed computing - How to classify a failure detector?

I understand that failure detectors in asynchronous systems are basically classified as (eventually)perfect/(eventually)strong and how those classes are defined, but I kind of struggle to get the intuition behind it.Suppose I have a concrete implementation of a failure detector, which periodically listens for heartbeat messages from each process. If a process hasn't sent its heartbeat message for a while, the process will be added to a list of suspects until a message is received from the process.Now, how do I know which class is this implement...Read more

distributed computing - Is there a relationship between CRDTs and the RAFT protocol - or are they orthogonal?

Take the use case of a multi-player networked game. Instantly you have the problem of replicating and reconciling shared state across the network. There appear to be a multiple of tools aimed at aspect of this problem, and two in particular seem to overlap:Conflict-free Replicated Data Types (CRDTs) - used for The RAFT consensus algorithm - for choosing a transactional leader in a distributed network to help achieve consensus. My question is: Is there a relationship between CRDTs and the RAFT protocol - or are they orthogonal?...Read more

distributed computing - How to force condor to submit job to all nodes in the cluster?

I have a condor cluster with multiple nodes active.But when I submit a job, it only runs on a single node (i.e Master node). I'm aware that Condor automatically distributes job based on available resources. But what if I want to force condor to make use of all the nodes? Just for the sake of evaluating process time when running on multiple nodes vs single node? I have tried adding requirements = Machine == "hostname1" && Machine == "hostname2" in the submit file, but isn't working....Read more

distributed computing - How to ensure distribution of a heavy task to other nodes using dispy?

I'm currently performing computation of the factorial of 10 random numbers using dispy, which "distributes" the tasks to various nodes.However, if one of the computation is of the factorial of a large number let's say factorial(100), then if the that task takes a very long time, yet dispy runs it only on a single node.How do I make sure that dispy breaks down and distributes this task to other nodes, so that it doesn't take so much time?Here's the code that I have come up with so far, where the factorial of 10 random numbers is calculated and t...Read more

distributed computing - Which CRDTs can be used to implement a full-featured collaborative rich text editor?

I have been studying CRDTs and understand that they have been used to build collaborative editors, including Ritzy, TreeDoc, WOOT and Logoot.I'm interested in building such an editor, and need to know if CRDTs are known to be able to handle this problem in its generality.To elaborate: A rich text document (think html) has a tree structure, but the nodes are heterogeneous. There are block elements, inline elements, tables, lists and so on. Further, there may be styles and stylesheets (e.g. css) embedded in a document. Finally, undo is essential....Read more

distributed computing - Handling memory and processor limitations of nodes in Blockchain

In Hyperledger Fabric Network(or someother Blockchain platforms), it is using a distributed ledger to store blocks of transactions. So all the participant nodes will have to maintain a ledger to keep the same copy of the chain of blocks. Here as time passes there will be more and more transactions, then *how it is managing the data load on each node?* Some of the participant nodes might have limitations in memory and all, so how it is managing properly for its smooth working? Also some of the participants like Mobile devices may not have much p...Read more

distributed computing - What is the difference between Sequential Consistency and Eventual Consistency?

There are two definitions I found on the Internet:Sequential consistency -- the result of any execution is the same as if the operations of all the processors were executed in some sequential order, and the operations of each individual processor appear in this sequence in the order specified by its program.Eventual Consistency -- if no new updates are made to a given data item, eventually all accesses to that item will return the last updated value.The definitions are clear to me. However, I don't get when eventual consistency is not sequentia...Read more

Consistency for read from distributed databases

I have a set of databases, distributed across multiple locations in the network and for ex. one client that needs to store some data in that databases. I need to make sure my data will always be stored.I can't organize a replica set with sync/async replication as it will make me to connect to one master which is a point of failure, so I send data from the client to all databases I know. Apparently, one database can fail to store, so I am relying on other databases writes. In the end I get different data sets stored in DB's though these sets are...Read more

distributed computing - Is the CAP theorem a red herring?

I am told that I have to give up transactional guarantees in large distributed systems because the CAP theorem says I can't have it. I think this is wrong for the following reasons:Internet routing is amazingly reliable.The CAP theorem only applies to network partitions where two groups of live machines can't communicate.Almost all real network partitions consist of catastrophic failures or cases where one of the partitions is very small and the other is very large and the small one can just shut down.Machine failures by themselves do not const...Read more

distributed computing - Need a 9 char length unique ID

My application uses a 9 digit number (it can be alphanumeric also). I can start with any number and then increments it at the beginning. But my application is not a single instance application, so if I run this exe as another instance, it should increment the latest value and the previous instance should again increment the latest value when it needs that value. I mean at all time, the value should be latest incremented value among all the instances that I open.This is half of the problem. The other side is, exes can be run on any machine on th...Read more