book

Implementing CIFS: The Common Internet File System

Name: Implementing CIFS: The Common Internet File System
Author: Christopher R. Hertel
ISBN: 013047116X

by Christopher R. Hertel

August 2003

Intermediate to advanced

672 pages

13h 59m

English

Pearson

Read now

Unlock full access

Copyright
Dedication
Praise for Implementing CIFS
Bruce Perens’ Open Source Series
About Prentice Hall Professional Technical Reference
About the Cover Art
Foreword
Introduction: CIFS from Eight Miles High
0.1. First Impressions0.2. What is CIFS?0.2.1. A Recipe for Protocol Soup0.3. The CIFS Community0.3.1. Visiting the Network Neighborhood0.3.2. Community Collaborations0.4. Audience0.5. Scope0.6. Acknowledgements and Thanks0.6.1. The Book0.7. About the Author0.7.1. Quick Story0.8. License
I. NBT: NetBIOS over TCP/IP
1. A Short Bio of NetBIOS
1.1. NetBIOS and DOS: The Early Years
2. Speaking NetBIOS
2.1. Emulating “NetBIOS LANs”2.1.1. The NetBIOS Name Service2.1.2. The NetBIOS Datagram Service2.1.3. The NetBIOS Session Service2.2. Scope: The Final Frontier2.3. Thus Endeth the Overview

3. The Basics of NBT Implementation
3.1. You Got the Name, Look Up the Number3.1.1. Encoding NetBIOS Names3.1.2. Fully Qualified NBT Names3.1.3. Second Level Encoding3.1.4. Name Service Packet Headers3.1.5. The Query Entry3.1.6. Some Trouble Ahead3.1.7. Finally! A Simple Broadcast Name Query3.2. Interlude
4. The Name Service in Detail
4.1. NBT Names: Once More with Feeling4.1.1. Valid NetBIOS Name Characters4.1.2. NetBIOS Names within Scope4.1.3. Encoding and Decoding NBT Names4.2. NBT Name Service Packets4.2.1. Name Service Headers4.2.2. Name Service Question Records4.2.3. Name Service Resource Records4.3. Conversations with the Name Service4.3.1. Name Registration4.3.1.1. Broadcast Name Registration4.3.1.2. Unicast (NBNS) Name Registration4.3.1.3. M and H Node Name Registration4.3.1.4. Registering Multi-Homed Hosts4.3.2. Name Query4.3.2.1. Negative Query Response4.3.2.2. Positive Query Response4.3.2.3. The Redirect Name Query Response4.3.2.4. A Simple Name Query Revisited4.3.3. Name Refresh4.3.4. Name Release4.3.4.1. Name Release Response4.3.5. Node Status4.3.5.1. Node Status Response4.3.6. Name Conflict Demand4.3.6.1. Name Release Demand Revisited4.4. Enough Already
5. The Datagram Service in Detail
5.1. Datagram Distribution over Routed IP Internetworks5.2. The NBDD and the Damage Done5.3. Implementing a Workable Datagram Service5.3.1. Fragmenting Datagrams5.3.2. Receiving Datagrams5.3.3. Querying the NBDD5.3.4. The Second Least Well Understood Aspect of NBT
6. The Session Service in Detail
6.1. Session Service Header6.2. Creating an NBT Session6.3. Maintaining an NBT Session6.4. Closing an NBT Session
7. Where It All Went Wrong
7.1. The 0x1Dirty Little Secret7.2. Twenty-five IPs or Less7.3. Special Handling Required for 0x1B Names7.4. Alternate Name Resolution7.5. The Awful Truth
II. SMB: The Server Message Block Protocol
8. A Little Background on SMB
8.1. Getting Started8.2. NBT or Not NBT
9. An Introductory Tour of SMB
9.1. The Server Identifier9.2. The Directory Path9.3. The File9.4. The SMB URL9.5. Was That Trip Really Necessary?
10. First Contact: Reaching the Server
10.1. Interpreting the Server Identifier10.2. The Destination Port10.3. Transport Discovery10.3.1. Run Naked10.3.2. Using the NetBIOS Name10.3.3. Reverse Mapping a NetBIOS Name10.4. Connecting to the Server
11. SMB in Its Natural Habitat
11.1. Our Very First Live SMBs11.2. SMB Message Structure11.2.1. SMB Message Header11.2.2. SMB Message Parameters11.2.3. SMB Message Data11.3. Case in Point: NEGOTIATE PROTOCOL11.4. The AndX Mutation11.5. The Flow of Conversation11.6. A Little More Code11.7. Take a Break
12. The SMB Header in Detail
12.1. The SMB_HEADER.STATUS Field Exposed12.2. The FLAGS and FLAGS2 Fields Tell All12.3. EXTRA! EXTRA! Read All About It!12.4. TID and UID: Separated at Birth?12.5. PID and MID Revealed12.5.1. EXTRA.PidHigh Dark Secrets Uncovered12.6. SMB Header Final Report
13. Protocol Negotiation
13.1. A Smattering of SMB Dialects13.2. Greetings: The NEGOTIATE PROTOCOL REQUEST13.3. Gesundheit: The NEGOTIATE PROTOCOL RESPONSE13.3.1. NegProt Response Parameters13.3.2. NegProt Response Data13.4. Are We There Yet?
14. Session Setup
14.1. SESSION SETUP ANDX REQUEST Parameters14.1.1. Virtual Circuits14.1.2. Capabilities Bits14.2. SESSION SETUP ANDX REQUEST Data14.3. The SESSION SETUP ANDX RESPONSE SMB
15. Authentication
15.1. Anonymous and Guest Login15.2. Plaintext Passwords15.2.1. User Level Security with Plaintext Passwords15.2.2. Share Level Security with Plaintext Passwords15.3. LM Challenge/Response15.3.1. DES15.3.2. Creating the Challenge15.3.3. Creating the LM Hash15.3.4. Creating the LM Response15.3.5. LM Challenge/Response: Once More with Feeling15.4. NTLM Challenge/Response15.5. NTLM Version 215.5.1. The NTLMv2 Toolbox15.5.2. The NTLMv2 Password Hash15.5.3. The NTLMv2 Response15.5.4. Creating the Blob15.5.5. Improved Security Through Confusion15.5.6. Insult to Injury: LMv215.5.7. Choosing NTLMv215.6. Extended Security: That Light at the End of the Tunnel15.6.1. The Extended Security Authentication Toolkit15.7. Kerberos15.8. Random Notes on W2K and NT Domain Authentication15.8.1. A Quick Look at W2K Domains15.8.2. A Few Notes about NT Domains15.8.3. It’s Good to Have a Backup15.8.4. Trust Me on This15.9. Random Notes on Message Authentication Codes15.9.1. Generating the Session Key15.9.2. Sequence Numbers15.9.3. Calculating the MAC15.9.4. Enabling and Requiring MAC Signing15.10. Non Sequitur Time15.11. Further Study
16. Building Your SMB Vocabulary
16.1. That TREE CONNECT Thingy16.2. SMB Echo16.3. Readin’, Writin’, and ’Rithmatic16.4. Transaction SMBs16.4.1. Mailslots and Named Pipes
17. The Remaining Oddities
17.1. Opportunistic Locks (OpLocks)17.1.1. OpLock Breaks17.2. Distributed File System (DFS)17.3. DOS Attributes, Extended File Attributes, Long Filenames, and Suchlike
18. That Just about Wraps Things Up for SMB
III. The Browse Service
19. A Beautiful Day in the Network Neighborhood
19.1. History: From Frontier Town to Bustling Metropolis19.2. Sociology19.3. Politics19.3.1. When Is a Workgroup not a Workgroup?19.3.2. Delegating Responsibility
20. Meet the Neighbors
20.1. Browse Service Clientele20.1.1. Providers20.1.2. Consumers20.2. The Local Master Browser20.3. Becoming a Backup Browser20.4. Crossing the Street with the DMB20.5. Elections
21. Infrastructure: The Mailslot and Named Pipe Abstractions
21.1. Meet the Plumbing: Named Pipes21.2. The Mailslot Metaphor
22. The Talk on the Street
22.1. Making Sense of SMBtrans22.2. Browse Service Mailslot Messages22.2.1. Announcement Request22.2.2. Host Announcement22.2.3. Election Request22.2.4. Get Backup List Request22.2.5. Get Backup List Response22.2.6. Local Master Announcement22.2.7. Master Announcement22.2.8. Domain Announcement22.2.9. Become Backup Request22.2.10. The Undocumented Reset22.2.11. It’s All in the Delivery22.3. RAPture22.3.1. NetServerEnum2 Request22.3.2. NetServerEnum2 Reply22.3.3. On the Outskirts of Town22.3.4. Transaction Fragmentation22.3.5. RAP Annoyances
23. The Better Browser Bureau
23.1. Running an Election23.1.1. Voting23.1.2. The Ballot23.2. Timing Is Everything
24. Samba Browse Service Enhancements
24.1. Automatic LANMAN24.2. UnBrowsable24.3. NBNS Wildcard DMB Queries and Enhanced Browsing24.4. Remote Announce24.5. Remote Browse Sync24.6. DMB != PDC
25. It Can’t Happen Here
25.1. Misconfigured Hosts25.2. Misconfigured Networks25.3. Implementation Bugs25.4. Troublemakers25.5. Design Flaws
26. At Home in the Network Neighborhood
IV. Appendices
A. Making a Good Cup of Tea
A.1. Basics of Making TeaA.2. About TeaA.3. Nasty HabitsA.4. Decaffeinating Tea
B. Known NetBIOS Suffix Values
B.1. NetBIOS Name Suffix BytesB.2. Special Handling of NetBIOS Names in WINS
C. The SMB URL
C.1. The Origins of the SMB URLC.2. Of Round Pegs, Square Holes, and Big MalletsC.3. Form Versus FunctionC.4. Additional PartsC.5. A Simple SMB URL Parser
D. CIFS Technical Reference
AbstractIntended UsageDISCLAIMER OF WARRANTIES AND REPRESENTATIONSLIMITATION OF LIABILITYINTELLECTUAL PROPERTY RIGHTSCOPYRIGHT AND USAGE AGREEMENTAcknowledgementsTable of Contents1. Introduction1.1. Summary of features1.1.1. File access1.1.2. File and record locking1.1.3. Safe caching, read-ahead, and write-behind1.1.4. File change notification1.1.5. Protocol version negotiation1.1.6. Extended attributes1.1.7. Distributed replicated virtual volumes1.1.8. Server name resolution independence1.1.9. Batched requests1.1.10. Obsolescence2. Protocol Operation Overview2.1. Server Name Determination2.2. Server Name Resolution2.3. Sample Message Flow2.4. CIFS Protocol Dialect Negotiation2.5. Message Transport2.5.1. Connection Management2.6. Opportunistic Locks2.6.1. Oplock Types2.6.1.1. Exclusive and Batch Oplocks2.6.1.2. Level II Oplocks2.6.2. Comparison with Other File Locking Methods2.6.3. Oplock SMBs2.6.3.1. Obtaining an Oplock2.6.3.2. Releasing an Oplock2.6.3.3. Revoking an Oplock2.6.4. Other Issues2.7. Security Model2.8. Authentication2.8.1. Overview2.8.2. Base Algorithms2.8.3. Authentication Algorithms2.8.3.1. NT Session Key2.8.3.2. LM Session Key2.8.3.3. Response2.8.3.4. MAC key2.8.3.5. Message Authentication Code2.8.4. Session Authentication Protocol2.8.4.1. Plain Text Password2.8.4.2. Challenge/Response2.8.5. Message authentication code2.8.6. Security Level2.9. Distributed File System (DFS) Support3. SMB Message Formats and Data Types3.1. Notation3.2. SMB header3.2.1. Command field3.2.2. Flags field3.2.3. Flags2 Field3.2.4. Tid Field3.2.5. Pid Field3.2.6. Uid Field3.2.7. Mid Field3.2.8. Status Field3.2.9. Timeouts3.2.10. Data Buffer (BUFFER) and String Formats3.3. Name Restrictions3.4. File Names3.5. Wildcards3.6. DFS Pathnames3.7. Time And Date Encoding3.8. Access Mode Encoding3.9. Access Mask Encoding3.10. Open Function Encoding3.11. Open Action Encoding3.12. File Attribute Encoding3.13. Extended File Attribute Encoding3.14. Batching Requests (“AndX” Messages)3.15. “Transaction” Style Subprotocols3.15.1. SMB_COM_TRANSACTION2 Format3.15.2. SMB_COM_NT_TRANSACTION Formats3.15.3. Functional Description3.15.4. SMB_COM_TRANSACTION Operations3.15.4.1. Mail Slot Transaction Protocol3.15.4.2. Server Announcement Mailslot Transaction3.15.4.3. Named Pipe Transaction Protocol3.15.4.4. CallNamedPipe3.15.4.5. WaitNamedPipe3.15.4.6. PeekNamedPipe3.15.4.7. GetNamedPipeHandleState3.15.4.8. SetNamedPipeHandleState3.15.4.9. GetNamedPipeInfo3.15.4.10. TransactNamedPipe3.15.4.11. RawReadNamedPipe3.15.4.12. RawWriteNamedPipe3.16. Valid SMB Requests by Negotiated Dialect4. SMB Requests4.1. Session Requests4.1.1. NEGOTIATE: Negotiate Protocol4.1.1.1. Errors4.1.2. SESSION_SETUP_ANDX: Session Setup4.1.2.1. Pre NT LM 0.124.1.2.2. NT LM 0.124.1.2.3. Errors4.1.3. LOGOFF_ANDX: User Logoff4.1.3.1. Errors4.1.4. TREE_CONNECT_ANDX: Tree Connect4.1.4.1. Errors4.1.5. TREE_DISCONNECT: Tree Disconnect4.1.5.1. Errors4.1.6. TRANS2_QUERY_FS_INFORMATION: Get File System Information4.1.6.1. SMB_INFO_ALLOCATION4.1.6.2. SMB_INFO_VOLUME4.1.6.3. SMB_QUERY_FS_VOLUME_INFO4.1.6.4. SMB_QUERY_FS_SIZE_INFO4.1.6.5. SMB_QUERY_FS_DEVICE_INFO4.1.6.6. SMB_QUERY_FS_ATTRIBUTE_INFO4.1.6.7. SMB_QUERY_CIFS_UNIX_INFO4.1.6.8. SMB_QUERY_MAC_FS_INFO4.1.6.9. Errors4.1.7. ECHO: Ping the Server4.1.7.1. Errors4.1.8. NT_CANCEL: Cancel request4.2. File Requests4.2.1. NT_CREATE_ANDX: Create or Open File4.2.1.1. Errors4.2.2. NT_TRANSACT_CREATE: Create or Open File with EAs or SD4.2.2.1. Errors4.2.3. CREATE_TEMPORARY: Create Temporary File4.2.3.1. Errors4.2.4. READ_ANDX: Read Bytes4.2.4.1. Errors4.2.5. WRITE_ANDX: Write Bytes to file or resource4.2.5.1. Errors4.2.6. LOCKING_ANDX: Lock or Unlock Byte Ranges4.2.6.1. Errors4.2.7. SEEK: Seek in File4.2.7.1. Errors4.2.8. FLUSH: Flush File4.2.8.1. Errors4.2.9. CLOSE: Close File4.2.9.1. Errors4.2.10. CLOSE_AND_TREE_DISCONNECT4.2.10.1. Errors4.2.11. DELETE: Delete File4.2.11.1. Errors4.2.12. RENAME: Rename File4.2.12.1. Errors4.2.13. NT_RENAME:4.2.13.1. Errors4.2.14. MOVE: Rename File4.2.14.1. Errors4.2.15. COPY: Copy File4.2.15.1. Errors4.2.16. TRANS2_QUERY_PATH_INFORMATION: Get File Attributes Given Path4.2.16.1. SMB_INFO_STANDARD & SMB_INFO_QUERY_EA_SIZE4.2.16.2. SMB_INFO_QUERY_EAS_FROM_LIST & SMB_INFO_QUERY_ALL_EAS4.2.16.3. SMB_INFO_IS_NAME_VALID4.2.16.4. SMB_QUERY_FILE_BASIC_INFO4.2.16.5. SMB_QUERY_FILE_STANDARD_INFO4.2.16.6. SMB_QUERY_FILE_EA_INFO4.2.16.7. SMB_QUERY_FILE_NAME_INFO4.2.16.8. SMB_QUERY_FILE_ALL_INFO4.2.16.9. SMB_QUERY_FILE_ALT_NAME_INFO4.2.16.10. SMB_QUERY_FILE_STREAM_INFO4.2.16.11. SMB_QUERY_FILE_COMPRESSION_INFO4.2.16.12. SMB_QUERY_FILE_UNIX_BASIC4.2.16.13. SMB_QUERY_FILE_UNIX_LINK4.2.16.14. SMB_MAC_DT_GET_APPL4.2.16.15. SMB_MAC_DT_GET_ICON4.2.16.16. SMB_MAC_DT_GET_ICON_INFO4.2.16.17. Errors4.2.17. TRANS2_QUERY_FILE_INFORMATION: Get File Attributes Given FID4.2.18. TRANS2_SET_PATH_INFORMATION: Set File Attributes given Path4.2.18.1. SMB_INFO_STANDARD & SMB_INFO_QUERY_EA_SIZE4.2.18.2. SMB_INFO_QUERY_ALL_EAS4.2.18.3. SMB_SET_FILE_UNIX_BASIC4.2.18.4. SMB_SET_FILE_UNIX_LINK4.2.18.5. SMB_SET_FILE_UNIX_HLINK4.2.18.6. SMB_MAC_SET_FINDER_INFO4.2.18.7. SMB_MAC_DT_ADD_APPL4.2.18.8. SMB_MAC_DT_REMOVE_APPL4.2.18.9. SMB_MAC_DT_ADD_ICON4.2.18.10. Errors4.2.19. TRANS2_SET_FILE_INFORMATION: Set File Attributes Given FID4.2.19.1. SMB_FILE_BASIC_INFO4.2.19.2. SMB_FILE_DISPOSITION_INFO4.2.19.3. SMB_FILE_ALLOCATION_INFO4.2.19.4. SMB_FILE_END_OF_FILE_INFO4.2.19.5. Errors4.3. Directory Requests4.3.1. TRANS2_CREATE_DIRECTORY: Create Directory (with optional EAs)4.3.1.1. Errors4.3.2. DELETE_DIRECTORY: Delete Directory4.3.2.1. Errors4.3.3. CHECK_DIRECTORY: Check Directory4.3.3.1. Errors4.3.4. TRANS2_FIND_FIRST2: Search Directory using Wildcards4.3.4.1. SMB_INFO_STANDARD4.3.4.2. SMB_INFO_QUERY_EA_SIZE4.3.4.3. SMB_INFO_QUERY_EAS_FROM_LIST4.3.4.4. SMB_FIND_FILE_DIRECTORY_INFO4.3.4.5. SMB_FIND_FILE_FULL_DIRECTORY_INFO4.3.4.6. SMB_FIND_FILE_BOTH_DIRECTORY_INFO4.3.4.7. SMB_FIND_FILE_NAMES_INFO4.3.4.8. SMB_FIND_FILE_UNIX4.3.4.9. SMB_FINDBOTH_MAC_HFS_INFO4.3.4.10. Errors4.3.5. TRANS2_FIND_NEXT2: Resume Directory Search Using Wildcards4.3.5.1. Errors4.3.6. FIND_CLOSE2: Close Directory Search4.3.6.1. Errors4.3.7. NT_TRANSACT_NOTIFY_CHANGE: Request Change Notification4.3.7.1. Errors4.4. DFS Operations4.4.1. TRANS2_GET_DFS_REFERRAL: Retrieve Distributed Filesystem Referral4.4.1.1. Errors4.4.2. TRANS2_REPORT_DFS_INCONSISTENCY: Inform a server about DFS Error4.4.2.1. Errors4.5. Miscellaneous Operations4.5.1. NT_TRANSACT_IOCTL4.5.1.1. Errors4.5.2. NT_TRANSACT_QUERY_SECURITY_DESC4.5.2.1. Errors4.5.3. NT_TRANSACT_SET_SECURITY_DESC4.5.3.1. Errors5. SMB Symbolic Constants5.1. SMB Command Codes5.2. SMB_COM_TRANSACTION2 Subcommand codes5.3. SMB_COM_NT_TRANSACTION Subcommand Codes5.4. SMB Protocol Dialect Constants6. Error Codes and Classes7. Security Considerations8. References9. Appendix A -- NETBIOS transport over TCP9.1. Connection Establishment9.2. Connecting to a server using the NetBIOS name9.3. Connecting to a server using a DNS name or IP address9.3.1. NetBIOS Adapter Status9.3.2. Generic Server Name9.3.3. - Parsing the DNS Name (guessing)9.4. NetBIOS Name character set10. Appendix B -- TCP transport11. Appendix C – Share Level Server Security12. Appendix D – CIFS UNIX Extension12.1. Introduction12.2. Principles12.3. CIFS Protocol Modifications12.4. Modified SMBs12.5. Guidelines for implementers13. Appendix E – CIFS Macintosh Extension13.1. Introduction13.2. Principles13.3. CIFS Protocol Modifications13.4. Modified SMBs13.5. Guidelines for implementers14. Appendix F – API Numbers for Transact based RAP calls
Glossary
References
BooksWeb

Content preview from Implementing CIFS: The Common Internet File System

Chapter 16. Building Your SMB Vocabulary

	...they have weapons of mass confusion and aren’t afraid to use them.
	--iomud on Slashdot

Looking back over our shoulders, we see that we have performed only two SMB exchanges so far: the NEGOTIATE PROTOCOL and the SESSION SETUP. There may be a TREE CONNECT shoved into the packet with the SESSION SETUP as an AndX, but we haven’t really described the TREE CONNECT in detail.

So, although we have covered a tremendous amount of material, our progress seems rather pathetic doesn’t it? What if the rest of SMB is just as tedious, verbose, and difficult?

Relax. It’s not.

Certainly there are other difficulties lying in wait, but the biggest ones have already been identified and we are carefully avoiding them. If you ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 013047116XPurchase book

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Implementing CIFS: The Common Internet File System

by Christopher R. Hertel

Chapter 16. Building Your SMB Vocabulary

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.