WriteString to TCP socket appears to be broken in Mathematica 12.3

Question

I have a large codebase in which some functions make calls to Python TCP servers. The code used to work perfectly in Mathematica 12.2, but has not worked since I updated to Mathematica 12.3 last night.

I have managed to isolate the problem to the call to WriteString. The following is a minimum illustration of the issue. Here is the Python server:

import socketserver
import json
import time


class TCPHandler(socketserver.BaseRequestHandler):
    def handle(self):
        bufSz = 2048
        while True:
            self.data = self.request.recv(bufSz)
            if self.data:
                jsonReq = json.loads(self.data.strip())
                print(f"Request: {jsonReq}")
                jsonRes = json.dumps({"res": len(jsonReq["arg"])})
                print(f"Response: {jsonRes}")
                self.request.sendall(bytes(jsonRes, "utf-8"))
            else:
                time.sleep(0.10)
                self.data = ""
                continue


def main():
    host, port = "localhost", 9990
    with socketserver.TCPServer((host, port), TCPHandler) as server:
        print(f"Test server now running at {host} on port {port}")
        server.serve_forever()


if __name__ == "__main__":
    main()

And here is the Wolfram Language code that calls it:

ClearAll[sock];
sock = SocketConnect[{"localhost", 9990}, "TCP"];

ClearAll[params];
params = ExportString[<|"arg" -> "test string"|>, "JSON"];

Module[{res},
    WriteString[sock, params];
    res = ByteArrayToString@SocketReadMessage[sock];
    ImportString[res, "JSON"]
]

This exact set-up used to work for me reliably until Mathematica 12.2. But now the execution hangs at the WriteString command.

I have tried turning off Windows firewall entirely. The problem persists.

Any assistance would be gratefully acknowledged.

Answer 1

I think there is a problem with your python TCP server implementation.

If you check the corresponding documentation, especially the example section, you may witness that the provided TCP client code does not always work either with your server.

client.py:

import socket
import sys

HOST, PORT = "localhost", 9990
data = " ".join(sys.argv[1:])

# Create a socket (SOCK_STREAM means a TCP socket)
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
    # Connect to server and send data
    sock.connect((HOST, PORT))
    sock.sendall(bytes(data + "\n", "utf-8"))

    # Receive data from the server and shut down
    received = str(sock.recv(1024), "utf-8")

print("Sent:     {}".format(data))
print("Received: {}".format(received))

run it with $ python3 client.py hello!. It sometimes hang on my machine (MacOS, Python 3.9.4).

Now if you replace your python server with the example implementation, it's possible to interact from the Wolfram Language.

This leads me to conclude that there must be a problem with your code sample. Could you double check this first, then we can investigate further what your project issue is. Let me provide few comments about the Python server to help debugging. First of all, why do you wrap the handler logic inside a while? I don't think it's necessary since the self.request.recv is a non busy wait. It may prevent correct connection handling. Also, why would you sleep if no data was received? Maybe that makes sense in your project, in which case disregard this comment, but in the example it seems unnecessary and confusing.

When I drop the while loop, and wrap the code inside a try / except, I'm able to communicate with a Wolfram Kernel. Here is a sample python server in case it helps:

import socketserver
import json
import time


class TCPHandler(socketserver.BaseRequestHandler):
    def handle(self):
        bufSz = 2048
        try:
            print("Waiting for a request...")
            self.data = self.request.recv(bufSz)
            print("Request received!")
            if self.data:
                jsonReq = json.loads(self.data.strip())
                print(f"Request: {jsonReq}")
                jsonRes = json.dumps({"res": len(jsonReq["arg"])})
                print(f"Response: {jsonRes}")
                self.request.sendall(bytes(jsonRes, "utf-8"))
            else:
                print("Empty request, sleeping.")
                time.sleep(0.10)
                self.data = ""
        except Exception as e:
            print("Error: ", e)

def main():
    host, port = "localhost", 9990
    with socketserver.TCPServer((host, port), TCPHandler) as server:
        print(f"Test server now running at {host} on port {port}")
        server.serve_forever()


if __name__ == "__main__":
    main()

Hopefully that will help you fixing your issue.