Date: 2014-07-09 OS: SLES OS Version: 11 SP1 Symantec FileStore 5.7 MP1 P1 Patch Upgrade README CONTENTS I. Overview II. Upgrade procedure III. Fixes in the new patch PATCH ID : N/A Release NAME : SFSFS-patch-5.7MP1P1_2014_05_30.tar.gz BASE PACKAGE NAME : Symantec FileStore BASE PACKAGE VERSION : 5.7 REQUIRED PATCHES : N/A SUPERCEDED PATCHES : 5.7MP1 INCOMPATIBLE PATCHES : N/A SUPPORTED OS : SLES SUPPORTED OS VERSION : SLES 11 SP1 CREATION DATE : 2014-07-09 CATEGORY : Enhancement, Bug Fix REBOOT REQUIRED : Yes SUPPORTS ROLLBACK : NO I. OVERVIEW: ------------ Symantec FileStore provides a scalable clustered storage solution. This document provides release information for the patch. Notes: 1. Patch upgrade will cause service failover from one node to another which will take about 15 seconds. This will not impact NFS client mounts and replication jobs, but CIFS client mounts will need to connect again, and backup jobs need to be resumed as well after an upgrade. 2. For the above reason, it is recommended to stop CIFS, backup, and EV I/O loads during an upgrade. 3. The 5.7MP1P1 patch can only be applied to the 5.7, 5.7P1, 5.7P2, 5.7P3 or 5.7MP1 release. 4. Patch upgrade cannot be rollbacked once applied. 5. Don't proceed upgrade when there are heavy I/O ongoing II. UPGRADE PROCEDURE: ---------------------- To install the patch: 1. Copy SFSFS-patch-5.7MP1P1_2014_05_30.tar.gz to the FileStore master node at (/home/support/patches/) using the support login. 2. If fencing is enabled, turn off and remove I/O fencing before the upgrade: filestore> storage> fencing off filestore> storage> fencing destroy 3. Install the FileStore patch using the master login (CLISH): filestore>upgrade patch install scp://support@localhost:/home/support/patches/SFSFS-patch-5.7MP1P1_2014_05_30.tar.gz The patch is installed on the slave nodes first followed by a system reboot. The master node is installed last followed by a system reboot. 4. If fencing was enabled, enable it again. 5. Monitor and check the status of the FileStore cluster: To check that the installation is complete, login to the master node again (switch to another node), and check that all the nodes are in the RUNNING state: filestore> cluster show Node State CPU(15 min) pubeth0(15 min) pubeth1(15 min) pubeth2(15 min) pubeth3(15 min) pubeth4(15 min) pubeth5(15 min) % rx(MB/s) tx(MB/s) rx(MB/s) tx(MB/s) rx(MB/s) tx(MB/s) rx(MB/s) tx(MB/s) rx(MB/s) tx(MB/s) rx(MB/s) tx(MB/s) ---- ----- ----------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- -------- filestore_01 RUNNING 45.44 1.63 0.01 4.87 0.04 5.46 0.02 0.00 0.00 0.00 0.00 0.00 0.00 filestore_02 RUNNING 28.40 1.63 0.01 4.60 0.03 10.94 0.18 0.00 0.00 0.00 0.00 0.00 0.00 filestore_01# gabconfig -a GAB Port Memberships ================================= Port a gen 187002 membership 01 Port b gen 187004 membership 01 Port f gen 18700e membership 01 Port h gen 187006 membership 01 Port u gen 18700c membership 01 Port v gen 187008 membership 01 Port w gen 18700a membership 01 Note: Please refer to the Symantec FileStore 5.7 Command-Line Administrator"s Guide for information on the "support" user login and more information about the patch installation. III. FIXES IN THE NEW PATCH: ---------------------------- Etrack Incidents: 2523199, 2597476, 2619873, 2641016, 2714958, 2718970, 2755442, 2765948, 2766474, 2780946, 2790990, 2813652, 2818809, 2822162, 2846009, 2869118, 2880859, 2882545, 2882615, 2883898, 2895870, 2895999, 2901712, 2904116, 2913690, 2915377, 2921154, 2921274, 2927986, 2928984, 2929184, 2931676, 2931724, 2933760, 2934660, 2935679, 2936482, 2937140, 2938006, 2944216, 2945525, 2945997, 2946907, 2946960, 2947066, 2951065, 2953550, 2954281, 2957132, 2959132, 2966506, 2967035, 2967311, 2968319, 2970148, 2972864, 2973245, 2973603, 2974925, 2979971, 2986819, 2988078, 2988561, 3000403, 3000769, 3000774, 3000779, 3000788, 3000793, 3003446, 3011068, 3011105, 3012451, 3018789, 3019755, 3024859, 3040836, 3041971, 3042743, 3046646, 3049172, 3050732, 3050776, 3050783, 3050799, 3050802, 3050873, 3055155, 3055973, 3058926, 3060455, 3064349, 3064355, 3067223, 3090640, 3091517, 3091614, 3093908, 3095771, 3101961, 3102083, 3109817, 3113334, 3121142, 3121143, 3121189, 3121205, 3121227, 3121239, 3121248, 3121258, 3121278, 3121282, 3121714, 3121719, 3121730, 3121737, 3121738, 3127123, 3128285, 3128611, 3132739, 3134660, 3138364, 3139905, 3139906, 3142731, 3162986, 3193747, 3198320, 3204775, 3211029, 3214311, 3217599, 3228830, 3252173, 3258309, 3258319, 3259658, 3260023, 3261727, 3263257, 3274671, 3280272, 3286871, 3286957, 3291761, 3295801, 3297485, 3299256, 3299525, 3301152, 3309500, 3309608, 3314356, 3316734, 3318720, 3320932, 3321232, 3323078, 3334554, 3339130, 3344343, 3344948, 3345351, 3475720 Errors/Problems Fixed: 2523199 Data satisfied the conditions of migration in secondary tier cannot be automatically trigger migration 2597476 `CLI>support debuginfo upload all` could trigger disk i/o error, "SysRq show state" seemed to cause it. snapshots disappered by this. 2619873 Support for bonding of Private NICs 2641016 `fsppadm enforce` spends too much time for relocate files from Tier1 to Tier2, time grows every day, policy has relocate rule only for Tier1 to Tier2. 2714958 /var/log/sfsfs_event_dup.log is broken by antivirus "could not scan.." messages. 2718970 FS destroy failed and replication job disable behavior 2755442 After `cli>cluster reboot all` two in 16 filesystems ended to FAULTED. system seems not be able to access to these two if they have "ckpt async operations". 2765948 Logoff from Windows7 client taking much time with FileStore hosting redirected folders for user profile 2766474 Full fsck flag is set in vx_cfs_iread(cont'd from e2702870) 2780946 While config export (Backup sfs script) should also backup the authorized keys from the clients so after upgrade, the nascli will work without having to seup the password free access again 2790990 Checkpoint removal takes long time if the file system has large number of files. 2813652 "storage fs shrinkby" command after creating the rollback shows improper message 2818809 File system is marked bad after an inode table overflow error. 2822162 Writes from nfs client get very slow when `nfs share add other-fs` is started, it is ~6-10 times slower. 2846009 Changes done to use cds format 2869118 `fsppadm enforce` running on multile nodes is needed for tier jobs to complete during night safely. 2880859 When optical fiber pulled out from the Master node of cluster, IO got hung on all the node and the disk utilization is 100% 2882545 Most of the CLISH commands fail, if the file system name contains "-"(hyphen). 2882615 `cli> antivirus autoprotect enable` mishandles fs-name involving "-"(hyphen) 2883898 5.7MP1 will upgrade vxvm, vxvm install script will set default dg to "nodg" instead of "sfsdg", so File system can't be destroyed 2895870 After upgrade the filestore from 5.7 to 5.7MP1, not able to connect to the server using ssh. 2895999 rtvscand clean up does not remove the /var/run/symantec/rtvscand.pid 2901712 CLI needs new `system option` to set longer value to OnlineTiemout of cfsmount resource 2904116 CLISH command "replication import key" unable handle default key file name in ipv6 environment. 2913690 Access to redirected folers shared by cifs of FileStore from Windows7 clinets were too slow. 2915377 `storage fs offline any-other-FS` causes replication sevice failover. 2921154 After adding vlan vip, vip does not come online automatically. 2921274 Replication bandwidth throttle not working with IPv6. 2927986 stat all showing a wrong cpu idle value 2928984 softlinks not created for NFS action agent action_NFS->filestore 2929184 ldap enable throws script errors. 2931676 Cannot start cifs server with clustering mode as ctdb. 2931724 after `storage fs set fullspace 85` and exceeded 85% full, but both nfs/cifs were not changed to readonly. 2933760 Command "report exportevents scp://support@sfsqa05_01:/var/ audit" showing script error on clish. 2934660 e-mail message of event notification shows incomplete message 2935679 Rate of DST enforce is slow when some 1G files are moving 2936482 System option modify ninodes fails to get lock while rebooting cluster. 2937140 [Replication] For destination filesystem, one of two nodes got faulted during switch-mount-point. Checking result-of-cfsumount should be done on both nodes. 2938006 Broadcom NetXtreme driver update needed in SFS 5.7P3HF1 2944216 Needs new funtion in autosnap_policy.sh to skip new snapshot creation when FS is having removing-job of no-named-fset. It's for protecting system form overload. 2945525 Use new ASLAPM for dorado 5100 array. 2945997 FileStore: Backup: ip routes are not added when backup ip is on different subnet than based device for IPv4 2946907 The Installation of file store may stopped with the hang as vxfs module removal got stuck. 2946960 While adding mirror to a filesystem, the migration speed is only 15MBps each LUN created from SATA disk 2947066 After setting up nbuclient and starting it , bp.conf missing "SANCLIENT=1" etc. "HostProperties" command from nbu-server fails. 2951065 Giving the same value for ninodes should throw warning instead of rebooting the node. 2953550 /opt/VRTSnasgw/scripts/fs_lib.sh: line 291: [: -ne: unary operator expected 2954281 When AD domain controller is abnormal, there will be many RST connection record saved under nlm volume and never been deleted 2957132 Replication is not working with multiple entry workset definitions 2959132 syntax error in frlpause_enable of nfs_agent.sh 2966506 vx_idetach generated kernel core dump while replication running. 2967035 Enhance OEM script for security related enhancement 2967311 space-optimized rollback creation failed due to not enough space 2968319 fsppadm enforce spends too much time on CVM slave node 2970148 GUI cannot delete snmp server. 2972864 syntax error in phantom_Phantomgroup_pubeth0.log for masktobits 2973245 addr_modify_output.log grows to over 80GB 2973603 lock() and lock2() using different lock file, but it share same unlock method unlock(), it might cause deadlock. 2974925 fsckptadm/fsppadm got frozen at vx_*() -> scheduled_timeout() on different two systems 2979971 Crash when destroying large file systems 2986819 Syntax error in fs_lib.sh 2988078 volume prepare failure during rollback create 2988561 mirroring filesystems result in all SFS commands locked 3000403 Enhance monitoring winbind 3000769 fs_alert.sh line 263 unary operator expected 3000774 disk detail path code path disk_list_paths.sh: line 114 has a script error 3000779 disk remove code path disk_lib.sh line number 520 has a script error 3000788 snapshot_lib.sh: line 1333: let: size=*0: syntax error: operand expected 3000793 syntax error while trying to display the striped-mirror volume 3003446 integrate snapshot best practice into CLISH operation 3011068 After adding the user, found diff in /etc/shadow file on nodes of cluster for encrypted password of newly added user. 3011105 Enhance import configuration for all the modules of FileStore so that this can be completed quickly. 3012451 `df -i` shows wrong/too-many number of inodes when FS has snapshots. 3018789 upgrade requires fencing off and on and this is effectively bringing in down time. Optimize this. 3019755 DMP is Not-Active 3024859 Nfsd lockup due to deadlock in deadlock between rename and lookup in VxFS 3040836 Disable volpagemod_max_memsz autotune during system boot. 3041971 For multiple cifs shares,`cli>storage fs alert set fullspace` works when set to read_only but it works partially when set read_only back to read_write. 3042743 Avoid fencing on and off during upgrade process. 3046646 the actually tmpfssize is not same as we configured 3049172 Antivirus job got hang due to FS locking 3050732 for multiple cifs shares,`cli>storage fs alert set fullspace` works when set to read_only but it works partially when set read_only back to read_write. 3050776 System panic when rebooted on SFS node 3050783 system stuck in kmem_cache_shrink() 3050799 Removing of filesystem from pool does not work if filesystem not in all disks ( need to specify individual disk ) 3050802 Rebooting mater node panics slave node in fencing enabled environment. 3050873 Replication: The FS going into offline state on it's own. 3055155 AntiVirus messages fill up email. The customer wants a solution in SFS 5.7MP1P1. 3055973 File names with special characters causes logging failure 3058926 Remove mirror fails. 3060455 Snapshot get offline after onlining them in sometime 3064349 Antivirus scan-action doesn't work 3064355 system panic in nfsd 3067223 some file could not be accessed with "No such device or address" on all CFS Slave nodes. 3090640 fs creation and snapshot destroying are hung. 3091517 Liveupdate deleted files under /tmp after patch installed 3091614 Files not read from buffer cache. 3093908 Provide more bonding options. 3095771 `/user/bin/snmptrap` in event_notify.sh occurs userName error for replication events, but it works for storage/etc events. 3101961 Increase max limit on ninodes. 3102083 When volpagemdo_maxmemsz is automatically tuned when there is a failure in cache object creation, throw an alert. 3109817 Segmentation fault caused by tmpfile doesnt exist in fsdedupadm 3113334 Fs destroy and create fails abruptly when logged in from an ssh client to FileStore node. 3121142 Upgrade is not preserving fs fullalerts, ninodes and volpagemod values. 3121143 Unable to Enable the job if remote cluster job wset entries same with source cluster job wset entries. 3121189 Remove op-nolock for NFS commands and import/export commands. 3121205 NFS import fails to import shares. 3121227 Find command not completing quickly and every time reading from disk. 3121239 adding disk to existing DG failing if its GPT 3121248 Upgrade failed as NFS service failed to online. 3121258 Though defined 288 nfsd threads, only 5 are effectively handling IO. 3121278 Provide fsadm defrag options to defrag only dirs or files or both 3121282 Performance issue while doing ls on an fs which got 39000 soft links. 3121714 kernel panic while doing NFS load. 3121719 Add mirror operation from SSH client hangs. 3121730 Optimize nfs share configuration import. 3121737 broken resource dependency if same shares exported multiple times to different clients. 3121738 /opt/VRTSnasgw/scripts/nfs_agent.sh: line 121: $output_fd: ambiguous redirect 3127123 Storage GUI home page cannot be displayed 3128285 core-dump on 3-node cluster 3128611 Installation fails some times and throws error that "profile file is missing" with blue screen. 3132739 'cli> storage snapshot quota on|off' may reset usage value incorrectly, might go negative. 3134660 Too many info message in log file 3138364 Server stop does not work 3139905 storage fs shrinkto" does not finish even if it is performing for 24 hours or more. 3139906 Reboot one of two controller may cause "fullfsck flag set” 3142731 'cli>replication job enable' fails when 'cfsmount' in rep_cfsmount() returns ret=23. 3162986 Pick up the correct VxVM RPM. 3193747 NDMP's restore job fails sowing "ERROR: mapping file: /vx/wangfeng/50g/50g.dat: Cannot allocate memory." 3198320 Update RPMs to address security vulnerabilities CVE-2012-3912, CVE-2012-1126 through 1144, CVE-2011-3102, CVE-2012-1667, CVE-2012-2807, CVE-2012-1182, CVE-2012-2152, CVE-2012-2088, CVE-2012-2113, CVE-2012-2110, CVE-2012-2131, CVE-2012-2088, CVE-2012-2113, CVE-2012-2337, CVE-2012-0420, CVE-2012-3817. 3204775 Debuginfo is hanging 3211029 File was saved as null file on quota file system 3214311 Avoid kswapd issue 3217599 Update NFS for 5.7mp1p1. 3228830 `replication job enable job1` deletes line in crontab for "job10". this is one of name-mis-matching-issus. 3252173 SFS replication Error: Error in changing ownership. 3258309 Remove smb2 in 5.7MP1P1. 3258319 Pick up the package vxfshfad. 3259658 When upgrade from 5.7p1 to 5.7p3hf3 then to 5.7mp1p1, the __homedir_fullacl missed in smbglobal.conf. 3260023 Issues on install slave node 3261727 Replication logs are filled very quickly which makes debugging impossible. 3263257 IPV6:: Able to add same ip using different ipv6 ip format. 3274671 ipv6:: On ipv6 installed cluster able to login to support account even though, support user status is Disable. 3280272 On Linux volumes can be grown greater than 256TB but cannot be started 3286871 setting same value for system option ninodes would still restart all nodes. 3286957 The IP type of IPv6 version 57MP1P1(2013-08-01) is IPv4 not IPv6. 3291761 IPV6 :: After upgrade not able to mount nfs share with ipv6 address. 3295801 SFS:5.7MP1P1:updown_test:Rollback has been created but gave the "failed" return value with errors. 3297485 When tried to shrink a 52TB FS to 2.5 TB via Filestore, fullfsck flag was set, fsck took too long and customer decided to remake a new FS with 2.5TB 3299256 user core dump details not collected if its generated by rpc.mountd. 3299525 `cli>storage snapshot create|destroy` still has "-" hyphen-naming-issue. example, /vx/test and /vx/test-01. 3301152 fcntl F_SETLKW hang on cfs 3309500 After upgrade from 5.7->5.7P3->5.7MP1P1 found some ftp parameters are missing. 3309608 Upgraded from 5.7->5.7P3->5.7MP1P1 then tried to set antivirus Scan read, Scan write, Manual scan zipdepth, Auto protect scan zipdepth. Its value is not reflecting in o/p of "antivirus show". 3314356 Modify "nfsd" does not work properly 3316734 rep_rm_src_star core dumps 3318720 Not able to mount NFS shared FS on nfs client. FS were marked as fullfsck. 3320932 'stat fsio' shows error on the other node 3321232 "storage tier policy run fs1" failed with "SFS fs ERROR V-288-1415 Command exited due to unknown error" 3323078 "network bond priv-create" got stuck at 85% 3334554 Cluster reboot failed if it has priv nic bond 3339130 Tried to import configuration from 5.6RP1P4HF4 -> 5.7MP1P1.Failed to to import configuration with "config import all" option. 3344343 IPV6:: update_parameters.sh not updating/modifying gateway and able to see old ip addresses in "ip route show" command. 3344948 remove unused dir for installation. 3345351 After config import 5.6RP1P4HF4 -> 5.7MP1P1, replication service is failing to start with error message "SFS replication ERROR V-288-2210 Replication configuration file is not valid." 3475720 Update glibc, vxfs and vxvm rpms