What does it mean if my latency is too high?

This could be a sign that your cluster is undersized for the load that is being used to access your data.

Adding more CU to increase the data size and distribution of data on your cluster can help reduce latency. If you are already at a 4CU  database size then there may be other options and Services may be able to assist you in lowering latency.

Please see the following on increasing the CU for your database:
https://docs.datastax.com/en/astra/aws/doc/dscloud/astra/dscloudResizeCluster.html