Whenever I'm looking for something akin to mutability I can typically get by via storing things in downvalues of variables. Here is a simple version of your ciruclar buffer that just writes the values to the downvalues of a unique symbol, and instead of passing the actual values around pass a small object of the form: buffer[var,n,l], where var is the symbol used for storage, n is the current location, and l is the length of the buffer.

SetAttributes[buffer, HoldAll];
newBuffer[n_] := Unique[] // buffer[#, 1, n] &;

append[buffer[var_, n_, l_], value_] :=
 (var[n] = value; buffer[var, #, l] &@Mod[n + 1, l, 1])

read[buffer[var_, n_, l_], m_] /; 1 <= n <= l := var[Mod[m - n, l, 1]]

Then you can create a new buffer and fill it up with values:

bigBuffer = newBuffer[1*^7];
i = 1;
fullbuffer = Nest[append[#, i++] &, bigBuffer, 1*^7];

And then add new values as:

Do[fullbuffer = 
 append[fullbuffer, RandomInteger@99], {100}] // AbsoluteTiming
(* {0.001000, Null} *)

To compare your orignal test on my system is:

big = Range@1*^7;
Do[big = Append[Rest@big, RandomInteger@99], {100}] // AbsoluteTiming
{5.430311, Null}

With this implementation you'll still have a unnecessary copying of the object used to represent the buffer, but it's very small, and if you really wanted to avoid copying it, n could similarly be stored as a down-value of the symbol (using var["current"] for instance). This might be a good idea anyway, as that avoids spawning off different objects that point to different locations in the buffer.

Whenever I'm looking for something akin to mutability I can typically get by via storing things in downvalues of variables. Here is a simple version of your ciruclar buffer that just writes the values to the downvalues of a unique symbol, and instead of passing the actual values around pass a small object of the form: buffer[var,n,l], where var is the symbol used for storage, n is the current location, and l is the length of the buffer.

SetAttributes[buffer, HoldAll];
newBuffer[n_] := Unique[] // buffer[#, 1, n] &;

append[buffer[var_, n_, l_], value_] :=
 (var[n] = value; buffer[var, #, l] &@Mod[n + 1, l, 1])

read[buffer[var_, n_, l_], m_] /; 1 <= n <= l := var[Mod[m - n + 1, l, 1]]

Then you can create a new buffer and fill it up with values:

bigBuffer = newBuffer[1*^7];
i = 1;
fullbuffer = Nest[append[#, i++] &, bigBuffer, 1*^7];

And then add new values as:

Do[fullbuffer = 
 append[fullbuffer, RandomInteger@99], {100}] // AbsoluteTiming
(* {0.001000, Null} *)

To compare your orignal test on my system is:

big = Range@1*^7;
Do[big = Append[Rest@big, RandomInteger@99], {100}] // AbsoluteTiming
{5.430311, Null}

With this implementation you'll still have a unnecessary copying of the object used to represent the buffer, but it's very small, and if you really wanted to avoid copying it, n could similarly be stored as a down-value of the symbol (using var["current"] for instance). This might be a good idea anyway, as that avoids spawning off different objects that point to different locations in the buffer.

Whenever I'm looking for something akin to mutability I can typically get by via storing things in downvalues of variables. Here is a simple version of your ciruclar buffer that just writes the values to the downvalues of a unique symbol, and instead of passing the actual values around pass a small object of the form: buffer[var,n,l], where var is the symbol used for storage, n is the current location, and l is the length of the buffer.

SetAttributes[buffer, HoldAll];
newBuffer[n_] := Unique[] // buffer[#, 1, n] &;

append[buffer[var, n_, l_], value_] :=
 (varname[n] = value; buffer[var, #, l] &@Mod[n + 1, l, 1])

read[buffer[var, n_, l_], m_] /; 1 <= n <= l := var[Mod[m - n, l, 1]]

Then you can create a new buffer and fill it up with values:

bigBuffer = newBuffer[1*^7];
i = 1;
fullbuffer = Nest[append[#, i++] &, bigBuffer, 1*^7];

And then add new values as:

Do[fullbuffer = 
 append[fullbuffer, RandomInteger@99], {100}] // AbsoluteTiming
(* {0.001000, Null} *)

To compare your orignal test on my system is:

big = Range@1*^7;
Do[big = Append[Rest@big, RandomInteger@99], {100}] // AbsoluteTiming
{5.430311, Null}

With this implementation you'll still have a unnecessary copying of the object used to represent the buffer, but it's very small, and if you really wanted to avoid copying it, n could similarly be stored as a down-value of the symbol (using var["current"] for instance). This might be a good idea anyway, as that avoids spawning off different objects that point to different locations in the buffer.

Whenever I'm looking for something akin to mutability I can typically get by via storing things in downvalues of variables. Here is a simple version of your ciruclar buffer that just writes the values to the downvalues of a unique symbol, and instead of passing the actual values around pass a small object of the form: buffer[var,n,l], where var is the symbol used for storage, n is the current location, and l is the length of the buffer.

SetAttributes[buffer, HoldAll];
newBuffer[n_] := Unique[] // buffer[#, 1, n] &;

append[buffer[var_, n_, l_], value_] :=
 (var[n] = value; buffer[var, #, l] &@Mod[n + 1, l, 1])

read[buffer[var_, n_, l_], m_] /; 1 <= n <= l := var[Mod[m - n, l, 1]]

Then you can create a new buffer and fill it up with values:

bigBuffer = newBuffer[1*^7];
i = 1;
fullbuffer = Nest[append[#, i++] &, bigBuffer, 1*^7];

And then add new values as:

Do[fullbuffer = 
 append[fullbuffer, RandomInteger@99], {100}] // AbsoluteTiming
(* {0.001000, Null} *)

To compare your orignal test on my system is:

big = Range@1*^7;
Do[big = Append[Rest@big, RandomInteger@99], {100}] // AbsoluteTiming
{5.430311, Null}

With this implementation you'll still have a unnecessary copying of the object used to represent the buffer, but it's very small, and if you really wanted to avoid copying it, n could similarly be stored as a down-value of the symbol (using var["current"] for instance). This might be a good idea anyway, as that avoids spawning off different objects that point to different locations in the buffer.