Central Services Operations corner

Links

Puppet templates: https://gitlab.cern.ch/ai/it-puppet-hostgroup-vopanda
PanDA configurations: https://gitlab.cern.ch/ai/it-puppet-module-vopandaconfig
CSOps managed machines: https://atlas-adcmon.cern.ch/cmdb/

Commands

Disable, enable, run puppet with logging in the foreground:

puppet agent --disable 'reason for disabling puppet'
puppet agent --enable
puppet agent -t

Take a PanDA server out of load balancing and put it back in:

touch /etc/iss.nologin  # take out of load balancing
rm /etc/iss.nologin     # put back in load balancing

Check the status and configuration of the logrotate service:

systemctl list-timers | grep logrotate
cat /etc/systemd/system/logrotate.timer

Myproxy for PanDA ProxyCache

For ATLAS Robot proxy certificate of atlpilo1 and atlpilo2 are in use. Examples below are for atlpilo1.

CSOps manages an automatic renewal script that runs in acrontab of atlpilo1 or atlpilo2. You can see check them like this:

ssh root@<harvester instance>
su -l atlpilo1
/usr/sue/bin/kinit -kt /data/atlpilo1/keytab atlpilo1@CERN.CH
acrontab -l

...
#check and upload atlpilo2 proxy in myproxy
00 09 * * * lxplus-acron.cern.ch /afs/cern.ch/user/a/atlpilo2/.globus/renew_myproxy.sh > /afs/cern.ch/user/a/atlpilo2/my_proxy.log 2>&1
...

You can try to check or renew the proxy manually. In case myproxy is not installed, install it first:

yum install myproxy

Check myproxy info:

myproxy-info -s myproxy.cern.ch -l '/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=atlpilo1/CN=614260/CN=Robot: ATLAS Pilot1'

Reinitialize myproxy:

myproxy-init -s myproxy.cern.ch -x -Z '/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=pandasv1/CN=663551/CN=Robot: ATLAS Panda Server1' -d -k panda -c 4383 -t 0 -C ~/.globus/atlpilo1_latest_x509up.rfc.proxy -y ~/.globus/atlpilo1_latest_x509up.rfc.proxy;