Probabilistic vs Deterministic
Software systems are usually deterministic, meaning, for a given set of inputs, they always produce same set of outputs. This is what we are used to but when we use LLMs, certain parts of the solution become probabilistic. Meaning, it will not always produce the same set of outputs for a given set of inputs. This introduces a great number of difficulties when developing solutions using LLMs.
This uncertainty even could lead to unprecedented project failures. A PoC and an MVP which gave promising results could miserably fail in a production setup. So it is important to understand this complexity and deal with it in a pragmatic way from the initial stage of the solution development. And this found to be a common pitfall for many RAG solutions.
This uncertainty even could lead to unprecedented project failures. A PoC and an MVP which gave promising results could miserably fail in a production setup. So it is important to understand this complexity and deal with it in a pragmatic way from the initial stage of the solution development. And this found to be a common pitfall for many RAG solutions.