When I use DeleteDuplicates to delete equal rows of a sparse array, the output is not a sparse array anymore. In my application, the sparse arrays are huge but very sparse, so converting back and forth is quite annoying.

Is there a version of, or alternative to, DeleteDuplicates that always preserves sparse arrays? (i.e. without converting back and forth between dense and sparse representations)

  • $\begingroup$ You mean, you want to delete duplicate rows? $\endgroup$ Jun 16, 2023 at 12:00
  • $\begingroup$ Yes. But preferably still with the possibility to use a test function of choice (e.g. when two rows are equal up to sign) $\endgroup$
    – Gert
    Jun 16, 2023 at 13:43
  • 5
    $\begingroup$ You can avoid unsparsifying rows by doing SparseArray @ DeleteDuplicates[List @@ sparse] $\endgroup$
    – Carl Woll
    Jun 16, 2023 at 14:23
  • 4
    $\begingroup$ Deleting duplicate rows would change the dimensions of the array. That in turn would change the coordinate->value rules. You won't be preserving much of anything. So, if DeleteDuplicates did return a sparse array, you'd just have a whole other set of annoyances to deal with. Are you sure DeleteDuplicates is the operation you want? Maybe there is a more algebraic operation that you're looking for. $\endgroup$
    – lericr
    Jun 16, 2023 at 15:44

1 Answer 1


It often helps when the OP provides their own test case. But here goes, assuming any test case sufficiently illustrates the problem as we are implicitly invited to do:


sparse = SparseArray[
  Thread[RandomInteger[{1, 1000}, {100, 2}] -> 1], {1000, 1000}]
(*  150671984  *)
sparse[[DeleteDuplicatesBy[Range@Length@sparse, sparse[[#]] &]]]
(*  150671984  *)

Or per the OP's comment under the OP:

sparse = SparseArray[
  Thread[RandomInteger[{1, 1000}, {100, 2}] -> 
    RandomChoice[{-1, 1}, 100]], {1000, 1000}]
 DeleteDuplicatesBy[Range@Length@sparse, Abs[sparse[[#]]] &]]]

(Same dimensions since Abs[-1] == 1.

  • $\begingroup$ Somewhat inspired by this answer to a question of mine. Which I guess I still remember. $\endgroup$
    – Michael E2
    Jun 17, 2023 at 23:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.