[gpfsug-discuss] AFM weirdness

[Simon Thompson (IT Research Support)]

We're using an AFM cache from our HPC nodes to access data in another GPFS
cluster, mostly this seems to be working fine, but we've just come across
an interesting problem with a user using gfortran from the GCC 5.2.0
toolset.

When linking their code, they get a "no space left on device" error back
from the linker. If we do this on a node that mounts the file-system
directly (I.e. Not via AFM cache), then it works fine.

We tried with GCC 4.5 based tools and it works OK, but the difference
there is that 4.x uses ld and 5x uses ld.gold.

If we strike the ld.gold when using AFM, we see:

stat("program", {st_mode=S_IFREG|0775, st_size=248480, ...}) = 0
unlink("program")                       = 0
open("program", O_RDWR|O_CREAT|O_TRUNC|O_CLOEXEC, 0777) = 30
fstat(30, {st_mode=S_IFREG|0775, st_size=0, ...}) = 0
fallocate(30, 0, 0, 248480)             = -1 ENOSPC (No space left on
device)



Vs when running directly on the file-system:
stat("program", {st_mode=S_IFREG|0775, st_size=248480, ...}) = 0
unlink("program")                       = 0
open("program", O_RDWR|O_CREAT|O_TRUNC|O_CLOEXEC, 0777) = 30
fstat(30, {st_mode=S_IFREG|0775, st_size=0, ...}) = 0
fallocate(30, 0, 0, 248480)             = 0



Anyone seen anything like this before?

... Actually I'm about to go off and see if its a function of AFM, or
maybe something to do with the FS in use (I.e. Make a local directory on
the filesystem on the "AFM" FS and see if that works ...)

Thanks

Simon