3drefit doesn't have a remove attribute option for any attribute, including atlas_point_list. That's on my list of things to do, but for atlases, you can get around this by copying the atlas to a new dataset. That attribute won't get copied over. Also you can specify which atlases you do want with the environment variable AFNI_ATLAS_LIST. If you make it only a single atlas, only that one will be searched. You can also limit the template spaces that are searched with AFNI_TEMPLATE_SPACE_LIST.

If your goal is to speed up AFNI startup, then here are some other tips and tricks.

Use -noplugins to avoid loading any of the plugin objects (or limit the plugins to the ones you want and set AFNI_PLUGIN_PATH).

AFNI_MAX_1DSIZE - sets the maximum amount of 1D data to read from each file. I’m not sure why we read any at all, but try setting this to a small number, maybe 1. (0 goes back to default) (-no1D and similar won't read the 1D files at all)

AFNI_START_SMALL - loads the smallest dataset into the viewer on startup instead of the first anatomical type

AFNI_SIMPLE_HISTORY - the datasets get their histories propagated and multiplied with multiple input commands like 3dcalc. All the extra history gets loaded in and can accumulate to something significant. Remove history from datasets with 3drefit, but that information will be lost.

AFNI_STARTUP_LAYOUT - this specifies a default script. If you don’t have it set, then the default .afni.startup.script in your home directory ( ~/.afni.startup_script ) is executed and opens datasets and image windows. Make sure it isn’t doing something you don’t want to do or just remove it.