Is a current of approximately 40 μA at 1.8 kV normal for a custom AC withstanding voltage test jig using TOS5300 Series?

Yes, this measurement is probably normal.
This current is probably due to stray capacitance and falls within the acceptable range for this type of setup.
For exact tolerance values, read the user’s manual – chapter of Specifications – Note No. 4: Regarding ammeter and judgment accuracy.