cylc config shows ‘communication method = zmq’, so that isn’t where the problem is coming from. HOWEVER, running with --debug does turn up errors:
Sending DEBUG MODE xtrace to job.xtrace
Traceback (most recent call last):
File "/glade/u/home/bcash/.conda/envs/cylc8b1_v2/lib/python3.7/site-packages/cylc/flow/task_message.py", line 108, in send_messages
pclient = get_client(suite)
File "/glade/u/home/bcash/.conda/envs/cylc8b1_v2/lib/python3.7/site-packages/cylc/flow/network/client_factory.py", line 52, in get_client
return get_runtime_client(get_comms_method(), workflow, timeout=timeout)
File "/glade/u/home/bcash/.conda/envs/cylc8b1_v2/lib/python3.7/site-packages/cylc/flow/network/client_factory.py", line 47, in get_runtime_client
return SuiteRuntimeClient(workflow, timeout=timeout)
File "/glade/u/home/bcash/.conda/envs/cylc8b1_v2/lib/python3.7/site-packages/cylc/flow/network/client.py", line 126, in __init__
host, port, _ = get_location(suite)
File "/glade/u/home/bcash/.conda/envs/cylc8b1_v2/lib/python3.7/site-packages/cylc/flow/network/__init__.py", line 81, in get_location
host = get_fqdn_by_host(host)
File "/glade/u/home/bcash/.conda/envs/cylc8b1_v2/lib/python3.7/site-packages/cylc/flow/hostuserutil.py", line 248, in get_fqdn_by_host
return HostUtil.get_inst().get_fqdn_by_host(target)
File "/glade/u/home/bcash/.conda/envs/cylc8b1_v2/lib/python3.7/site-packages/cylc/flow/hostuserutil.py", line 154, in get_fqdn_by_host
return self._get_host_info(target)[0]
File "/glade/u/home/bcash/.conda/envs/cylc8b1_v2/lib/python3.7/site-packages/cylc/flow/hostuserutil.py", line 118, in _get_host_info
self._host_exs[target] = socket.gethostbyname_ex(target)
socket.gaierror: [Errno -2] Name or service not known: 'cheyenne5.cheyenne.ucar.edu'
/bin/sh: BASH_XTRACEFD: 19: invalid value for trace file descriptor
Traceback (most recent call last):
File "/glade/u/home/bcash/.conda/envs/cylc8b1_v2/lib/python3.7/site-packages/cylc/flow/task_message.py", line 108, in send_messages
pclient = get_client(suite)
File "/glade/u/home/bcash/.conda/envs/cylc8b1_v2/lib/python3.7/site-packages/cylc/flow/network/client_factory.py", line 52, in get_client
return get_runtime_client(get_comms_method(), workflow, timeout=timeout)
File "/glade/u/home/bcash/.conda/envs/cylc8b1_v2/lib/python3.7/site-packages/cylc/flow/network/client_factory.py", line 47, in get_runtime_client
return SuiteRuntimeClient(workflow, timeout=timeout)
File "/glade/u/home/bcash/.conda/envs/cylc8b1_v2/lib/python3.7/site-packages/cylc/flow/network/client.py", line 126, in __init__
host, port, _ = get_location(suite)
File "/glade/u/home/bcash/.conda/envs/cylc8b1_v2/lib/python3.7/site-packages/cylc/flow/network/__init__.py", line 81, in get_location
host = get_fqdn_by_host(host)
File "/glade/u/home/bcash/.conda/envs/cylc8b1_v2/lib/python3.7/site-packages/cylc/flow/hostuserutil.py", line 248, in get_fqdn_by_host
return HostUtil.get_inst().get_fqdn_by_host(target)
File "/glade/u/home/bcash/.conda/envs/cylc8b1_v2/lib/python3.7/site-packages/cylc/flow/hostuserutil.py", line 154, in get_fqdn_by_host
return self._get_host_info(target)[0]
File "/glade/u/home/bcash/.conda/envs/cylc8b1_v2/lib/python3.7/site-packages/cylc/flow/hostuserutil.py", line 118, in _get_host_info
self._host_exs[target] = socket.gethostbyname_ex(target)
socket.gaierror: [Errno -2] Name or service not known: 'cheyenne5.cheyenne.ucar.edu'
In addition to this error, when the ‘hello’ and ‘world’ tasks start up the following error is displayed in my terminal:
(cylc8b1_v2) bcash@cheyenne5:/glade/work/bcash/cylc/debug_comms> X11 connection rejected because of wrong authentication.
If I ping cheyenne5 from a login node ‘cheyenne5.cheyenne.ucar.edu’ is the name it returns, so the name isn’t malformed or something. Hopefully something in this is helpful!
EDIT: job.xtrace shows both:
+[20210608T064901-0600]bcash@r2i6n23 cylc message -- debug_comms/run7 1/hello/01 started
and later:
+[20210608T064941-0600]bcash@r2i6n23 cylc message -- debug_comms/run7 1/hello/01 succeeded