7
$\begingroup$

I have Window 10, Mathematica 11.3 and Visual Studio Community 2013 (Update 5). If I try to get the package CCOmpilerDriver`, I get follow error:

Needs["CCompilerDriver`"]

Import::utf8badlead: Input is not a valid UTF8 byte sequence. Continuation byte 129 is expected to be a leading byte.

Import::jsonhintposition: An error occurred at line 9:22

And if you execute such a line of code:

<<"CCompilerDriver`CCompilerDriverBase`"

Import::utf8badlead: Input is not a valid UTF8 byte sequence. Continuation byte 129 is expected to be a leading byte.

Import::jsonhintposition: An error occurred at line 9:22

FilterRules::rep: CCompilerDriver`CCompilerDriverBase`Private`CommonCompilerOptions is not a valid replacement rule.

Join::heads: Heads List and FilterRules at positions 1 and 2 are expected to be the same.

FilterRules::rep: CCompilerDriver`CCompilerDriverBase`Private`CommonCompilerOptions is not a valid replacement rule.

Join::heads: Heads List and FilterRules at positions 1 and 2 are expected to be the same.

FilterRules::rep: CCompilerDriver`CCompilerDriverBase`Private`CommonCompilerOptions is not a valid replacement rule.

General::stop: Further output of FilterRules::rep will be suppressed during this calculation.

Join::heads: Heads List and FilterRules at positions 1 and 2 are expected to be the same.

General::stop: Further output of Join::heads will be suppressed during this calculation.

How to fix it? An error in the code of the СCompilerDriver?

$\endgroup$
3
  • 3
    $\begingroup$ We have discussed this in chat a couple of weeks ago. I am convinced that this is a bug, so please do report it to Wolfram Support. I believe the bug is triggered only when Windows is set to use a non-English language. $\endgroup$
    – Szabolcs
    Commented Jun 29, 2018 at 9:37
  • 3
    $\begingroup$ See around here: chat.stackexchange.com/transcript/message/45172407#45172407 $\endgroup$
    – Szabolcs
    Commented Jun 29, 2018 at 9:39
  • $\begingroup$ I also thought that this was a mistake. But I can't even understand what the problem is $\endgroup$ Commented Jun 29, 2018 at 9:45

2 Answers 2

8
$\begingroup$

Please use the fix from @ilian's answer instead!!


The underlying problem

I think the underlying problem is with the RawJSON importer. JSON is required to be UTF8. But a character encoding like UTF8 only makes sense when reading from a file, not when reading from a string (using ImportString) that is already interpreted and doesn't have an encoding.

Here's a demonstration of what I believe is going wrong:

In[21]:= ImportString["\"jó\"", "RawJSON"]

During evaluation of In[21]:= Import::utf8incompletesequence: Input is not a valid UTF8 byte sequence. The final multibyte sequence is incomplete.

During evaluation of In[21]:= Import::jsonhintposandchar: An error occurred near character '"', at line 1:5

Out[21]= $Failed

The RawJSON importer seems to be interpreting the string as a byte-sequence instead of a character sequence.

We can change the string into a form that it will accept:

In[23]:= ExportString["\"jó\"", "Text", CharacterEncoding -> "UTF8"]    
Out[23]= "\"jÃ\.b3\""

Now it's happy:

In[24]:= ImportString[
 ExportString["\"jó\"", "Text", CharacterEncoding -> "UTF8"],
 "RawJSON"
 ]

Out[24]= "jó"

A possible workaround

I have not tested the following workaround as I am not using Windows at this moment, so I couldn't even reproduce the problem you see in a direct way.

However, I believe the following may work. Do this only with version 11.3.0 on Windows!!

Open the folder where the VS driver is using

SystemOpen@FileNameJoin[{$InstallationDirectory, "SystemFiles", "Components", "CCompilerDriver"}]

You should see the file VisualStudioCompiler.m. Make a backup and open it with a text editor. Find line 384, which should be:

$vsInfo := $vsInfo = If[$vswhereWorks, ImportString[$vswhereOutput, "RawJSON"], False]

Change it to

$vsInfo := $vsInfo = If[$vswhereWorks, ImportString[ExportString[$vswhereOutput, "Text", CharacterEncoding -> "UTF8"], "RawJSON"], False]

Restart Mathematica and try loading the C compiler driver again. Does it work now?

$\endgroup$
4
  • $\begingroup$ Many thanks! I also tried to find the problem in this file $\endgroup$ Commented Jun 29, 2018 at 11:12
  • $\begingroup$ @KirillBelov Did it work though? As I said, I cannot actually test this. $\endgroup$
    – Szabolcs
    Commented Jun 29, 2018 at 12:31
  • $\begingroup$ I can check it in a few hours and sure I will write here about results $\endgroup$ Commented Jun 29, 2018 at 12:53
  • $\begingroup$ It really solved the problem, once again many thanks!! $\endgroup$ Commented Jun 29, 2018 at 19:38
6
$\begingroup$

This has been fixed by a paclet update.

Windows 11.3 installations will be automatically updated over the next few days.

To get the new paclet immediately, try

PacletSiteUpdate /@ PacletSites[];
PacletUpdate["CCompilerDriver"]
$\endgroup$
6
  • $\begingroup$ Is it for Windows only? I was just curious about what fix was used (I'm on OS X). $\endgroup$
    – Szabolcs
    Commented Jun 29, 2018 at 13:02
  • $\begingroup$ Windows only. The fix uses a recent vswhere update by Microsoft that supports UTF-8 output. $\endgroup$
    – ilian
    Commented Jun 29, 2018 at 13:11
  • $\begingroup$ I think that ImportString[..., "RawJSON"] is still broken, though. It should not try to re-interpret characters in the string according to some encoding. A string is not a byte sequence but an (already interpreted) character sequence. $\endgroup$
    – Szabolcs
    Commented Jun 29, 2018 at 13:24
  • $\begingroup$ The importer insists on valid JSON, that means UTF-8. $\endgroup$
    – ilian
    Commented Jun 29, 2018 at 13:26
  • $\begingroup$ You're right that valid JSON file must be UTF8, but only a file (i.e. a byte sequence) has an encoding. A Mathematica string does not have an encoding. A string is a sequence of characters, not a sequence of bytes. $\endgroup$
    – Szabolcs
    Commented Jun 29, 2018 at 14:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.