We construct a binary market model with memory that approximates a continuous-time market model driven by a Gaussian process equivalent to Brownian motion. We give a sufficient condition for the binary model to be arbitrage-free. In a case when arbitrage opportunities exist, we present the rate at which the arbitrage probability tends to zero.