Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Block/disallow --net=host (host networking) on Mac OS #2716

Closed
vhosakot opened this issue Mar 22, 2018 · 98 comments
Closed

Block/disallow --net=host (host networking) on Mac OS #2716

vhosakot opened this issue Mar 22, 2018 · 98 comments

Comments

@vhosakot
Copy link

Description

https://docs.docker.com/network/network-tutorial-host/ says:

The host networking driver only works on Linux hosts, and is not supported on Docker for Mac

--net=host (host networking) does not work on Mac as it is not supported on Mac. https://forums.docker.com/t/should-docker-run-net-host-work/14215 also says that --net=host is not supported on Mac.

Hence, --net=host should be blocked/disallowed on Mac and an error like --net=host (host networking) is not supported on Mac should be displayed if --net=host is used on Mac. This way, users on Mac know that --net=host is not supported on Mac and will not waste time debugging host networking issues on Mac.

Steps to reproduce the issue:

  1. Run any docker container with --net=host (host networking) on Mac.
    docker run -d --name docker-nginx --net=host -p 80:80 nginx
    
  2. Check if the container's port can be accessed on the Mac host.
    $ curl localhost:80
    curl: (7) Failed to connect to localhost port 80: Connection refused
    

Describe the results you received:

Not able to access the container's port on the Mac host when --net=host is used on Mac.

Describe the results you expected:

$ docker run -d --name docker-nginx --net=host -p 80:80 nginx
ERROR: --net=host (host networking) is not supported on Mac.

Additional information you deem important:

Output of docker version:

$ docker version
Client:
 Version:	17.12.0-ce
 API version:	1.35
 Go version:	go1.9.2
 Git commit:	c97c6d6
 Built:	Wed Dec 27 20:03:51 2017
 OS/Arch:	darwin/amd64

Server:
 Engine:
  Version:	17.12.0-ce
  API version:	1.35 (minimum version 1.12)
  Go version:	go1.9.2
  Git commit:	c97c6d6
  Built:	Wed Dec 27 20:12:29 2017
  OS/Arch:	linux/amd64
  Experimental:	true

Output of docker info:

$ docker info
Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 1
Server Version: 17.12.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 89623f28b87a6004d4b785663257362d1658a729
runc version: b2567b37d7b75eb4cf325b77297b140ea686ce8f
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.60-linuxkit-aufs
Operating System: Docker for Mac
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.952GiB
Name: linuxkit-025000000001
ID: QWYO:HNLA:VPRN:ZTTG:YUNW:CO37:BWYB:L4Z5:CFDY:JC3H:A6DD:RQDM
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 29
 Goroutines: 49
 System Time: 2018-03-22T04:19:59.058199778Z
 EventsListeners: 2
HTTP Proxy: docker.for.mac.http.internal:3128
HTTPS Proxy: docker.for.mac.http.internal:3129
Registry: https://index.docker.io/v1/
Labels:
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details:

Mac OS Sierra (version 10.12.6)

@vhosakot
Copy link
Author

Opened this issue per suggestion in moby/moby#36666 (comment).

@pgayvallet
Copy link

As you probably know, on D4M, docker is running inside a VM.
So "-net=host is not working" is subject to interpretation : it's working in the fact that the container uses the VM network, it's not in the fact that the VM network is not the host network.

However, in some use case, the actual behaviour is the one expected. That's why we just can't simply disable the whole option on D4D, as it would break for some of our users using it.

The solution is probably more on documentation than on blocking the option.

@vhosakot
Copy link
Author

vhosakot commented Mar 23, 2018

@pgayvallet Thanks for the info. Yes, when --net=host is used on a Mac, good documentation and an example how to access the container's port on the VM network inside the VM on a Mac would be very helpful. The documentation could also include the step to setup port-forwarding on the Mac to the VM's port so that the container's port can be accessed directly from the Mac when --net=host is used.

Also, can you send me steps to access the container's port on the VM network inside the VM on Mac when --net=host is used? I was not able to find them online.

@jakirkham
Copy link

Looks related to issue ( #68 ).

@zhouxiangxiang
Copy link

I have spend days for debugging network issues on mac.
And now I found this issue.
It should post some information tell user it's not supported on mac

@docker-robott
Copy link
Collaborator

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

@0x1ee7
Copy link

0x1ee7 commented Aug 3, 2018

/remove-lifecycle stale

@sregger
Copy link

sregger commented Sep 28, 2018

As you probably know, on D4M, docker is running inside a VM.

Had no idea this was the case until I started searching for why

docker --net=host

was not working. This is a confusing User Experience and should be improved. Appreciate there is room for interpretation. But common sense usage should win out. I don't think that's currently the case.

Can this issue be re-opened?

@fgarcia
Copy link

fgarcia commented Nov 8, 2018

I always thought that there should be another option like --net=host-virtual or something like that. Then host would work only in Linux and host-virtual in non Linux environments. Failing on non supported option would encourage users to read the manual, not to explore github issues

@deftdawg
Copy link

I've probably just wasted 3 days trying to get 2 different applications (home-assistant and mythtv) that will never work without a real implementation of host network because I had guessed given that -p ports are published that the host networking should also work...

Please don't tell me host networking does work but only for the VM, host networking means binding directly to the physical network / physical NIC, the VM doesn't do that and I can't even figure out how to access the VM (docker-machine doesn't seem to do anything on OSX)...

I can't imagine how much time people have collectively wasted troubleshooting this!

Invoking docker with --net=host should fail with an ERROR when started from non-linux OSes.

@vhosakot
Copy link
Author

Very happy to see this issue is helping many people! 😄

@deftdawg
Copy link

@vhosakot This issue isn't really helping anyone... 😉

But anyway... For those struggling with this problem here's something that will actually help; I've been able to work around 'docker for mac's limitation by the following:

Since using a Vbox bridged nic puts the docker-machine host VM directly on the network at it's own IP, network=host works from there at it's new IP.

@elver
Copy link

elver commented Jan 29, 2019

This is the most absurdly simple "bug" I've seen where fixing it should take a developer literally minutes, but nobody has done it for over half a year.

@kazzkiq
Copy link

kazzkiq commented Mar 12, 2019

Users should at least be warned when trying to use networking in Docker since it's capped in macOS and Windows.

At least a message like:

[Docker Networking]:
We detected you're fiddling with networking, but sadly it's only supported in Linux.

would be enough.

Also, it's 2019. Honestly, buy some macs and PCs and enforce them on your team so they actually see the issues we're having. That would do wonders for QA.

@DonGiulio
Copy link

There's indeed some lack of documentation here,

is there a way I can connect to the docker virtual machine so that I could check things before I deploy to prod (which is a Linux system)?

@kojustin
Copy link

kojustin commented Apr 7, 2019

I just wasted a day of time thinking that this worked the same that it does on Linux.

@vhosakot
Copy link
Author

vhosakot commented Apr 8, 2019

I just wasted a day of time thinking that this worked

@kojustin Wasting just a day on this issue is very productive lol 😉

@mtlott
Copy link

mtlott commented Apr 14, 2019

And I just wasted 2 hours on this. So I guess I am more productive than the last guy.

But Really???!!???

I don't work with docker day in and out. There should be error or warning message on D4M that reminds about these nuances. My use case is usually picking up a working compose stack and bringing it locally to a Mac to make some tweaks. A little warning that says this one won't work here would have been nice.

@ghost
Copy link

ghost commented Apr 25, 2019

And I just wasted 2 hours on this. So I guess I am more productive than the last guy.

But Really???!!???

I don't work with docker day in and out. There should be error or warning message on D4M that reminds about these nuances. My use case is usually picking up a working compose stack and bringing it locally to a Mac to make some tweaks. A little warning that says this one won't work here would have been nice.

same

@danielo515
Copy link

Ok, It is very disappointing that this does not work, but I can understand it.
What it is very frustrating is to waste time thinking that the command you just launched works but in reality it does not.
At least a warning is required, please.

@Pasukaru
Copy link

I also spent a few hours on this. Once you know why, it makes sense.

However, it should definitely throw an error and not continue without any message whatsoever, making users (including me) think the command will work.

@jmo-qap
Copy link

jmo-qap commented Aug 31, 2022

@nicks on the scene! Thanks!

@danperks
Copy link

Very excited to hear there's some motion in the ocean on this! Subscribe to the thread, good luck brave devs!

@carlosbaraza
Copy link

What a sink of dev time for not implementing a warning, which would take 5 minutes to someone experienced with the docker for mac codebase... I was trapped like so many other people in this thread...

@cocaesprit
Copy link

At this point is getting comical

@shatacheruvu
Copy link

fixed all my sins all this while but this issue is still open.

@ItsTarik
Copy link

Is this a payed feature ? LOOL

@IonasElate
Copy link

I would actually pay for this (a solution, not a warning) ;-)

@kopax-polyconseil
Copy link

Lucky to learn it now after a few days configuring a platform install. Cannot do as planned, any plan to resolve this ? 😆

@andbi
Copy link

andbi commented Dec 28, 2022

During these five years I've turned 50, became a grandfather, survived COVID, and bought a house. Anyone else care to share their stories while the issue is being actively resolved? :-)

@kopax-polyconseil
Copy link

During these five years I've turned 50, became a grandfather, survived COVID, and bought a house. Anyone else care to share their stories while the issue is being actively resolved? :-)

It took me a bit less to understand this comment is outdated: #2716 (comment)

I have hit this error: https://forums.virtualbox.org/viewtopic.php?f=8&t=107680

Stderr: 0%...NS_ERROR_FAILURE
VBoxManage: error: Failed to create the host-only adapter
VBoxManage: error: Failed to execute '/Applications/VirtualBox.app/Contents/MacOS/VBoxNetAdpCtl add 2>&1' - exit status: 34304
VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component HostNetworkInterfaceWrap, interface IHostNetworkInterface
VBoxManage: error: Context: "RTEXITCODE handleCreate(HandlerArg *)" at line 105 of file VBoxManageHostonly.cpp

So there's definitely no way now to run a container with host network ?

@fwnan

This comment was marked as abuse.

@jwoodrow
Copy link

jwoodrow commented Jan 6, 2023

The only way I managed to get it to work is to run docker using devcontainers which somehow makes things work out. But this is ridiculous. I understand things aren't simple and you've had to deal with a lot of mac issues what with all the Apple Silicon changes.

But it can't honestly take 5 years to either:

  • add an error message on non-linux machines
  • Find a solution to make this work on macs

Docker is meant to be a solution to standardize how developers collaborate and run projects on whichever platform they want. But in reality that isn't the case because some options just won't work on other systems and rather than say so directly and reject the command you prefer to have your users rip their hair out of frustration debugging known issues.

@nicks
Copy link

nicks commented Jan 6, 2023

Making Host OS networking work on MacOS is tracked here: docker/roadmap#238

Blocking this with an error message is not really on the table, because there are lots of legitimate use-cases for binding to the Host VM network (rather than the Host OS network). For example, Telepresence uses --net=host and wants to bind to the VM network, here's a good post on it: https://www.getambassador.io/kubernetes-learning-center/telepresence-docker-extension

There's been some discussion about making it a warning or having a way to disambiguate like --net=host-os or --net=host-vm, but there are concerns about portability.

@deftdawg
Copy link

deftdawg commented Jan 6, 2023

There should've been a warning when Docker for Mac was first released. At least the command line docker command should warn something about it, even if the Docker for Mac GUI people can't figure out how to handle it.

Besides the 100 or so folks who've found this thread and commented, think of all the other people who give up trying to run a tutorial or a product installation on mac that includes --net=host because it appears to be silently broken.

It would take what to implement a warning? An OS check (docker mac cli), a docker version check (< whatever version will eventually implement it) and a printf?

Blocking this with an error message is not really on the table, because there are lots of legitimate use-cases for binding to the Host VM network (rather than the Host OS network). For example, Telepresence uses --net=host and wants to bind to the VM network, here's a good post on it: https://www.getambassador.io/kubernetes-learning-center/telepresence-docker-extension

There's been some discussion about making it a warning or having a way to disambiguate like --net=host-os or --net=host-vm, but there are concerns about portability.

@disposedtrolley
Copy link

cc @davidlopezre

@ryanmccartney
Copy link

Happy 5th Birthday to this issue 🍰

@vhosakot
Copy link
Author

yay, happy 5th Birthday all! 🍰

@pjamessteven
Copy link

pjamessteven commented Mar 30, 2023

Happy birthday for three days ago :( If I had the skill and/or time I would try and fix this, but alas I don't. I hope that one day this issue gets resolved by a majestic computer wizard.

@yrik
Copy link

yrik commented Jun 20, 2023

So, what are the workarounds for now?

@atancredi

This comment was marked as off-topic.

@bsnk-dev
Copy link

bsnk-dev commented Aug 7, 2023

If you want docker containers to be able to access a specific application running on say, 127.0.0.1:3000, one solution is to run a proxy inside the container to the host.docker.internal network bridge. You can do so with a utility called socat.

An example so the docker container can access 127.0.0.1:3000 on your machine:

apt-get update && apt-get install -y socat
socat TCP-LISTEN:3000,fork TCP:host.docker.internal:3000

@yafetnid

This comment was marked as off-topic.

@phiberber
Copy link

I'm almost sure the entire team unsubscribed from this Issue, thankfully I got to find out it doesn't work by using Docker Desktop and "lsof" seeing there was no port open, we're near the 6th birthday!

@eshaan-deepsource

This comment was marked as off-topic.

@cristiano182
Copy link

@danielo515Ok, obrigado pelo esclarecimento! Existe alguma solução alternativa? Preciso acessar 127.0.0.1da máquina host durante a construção do Docker.

EDITAR: Não importa. Acabei de descobrir que você pode usar http://host.docker.internal:<your-port>para acessar a máquina host em vez de http://127.0.0.1:<your-port>durante a construção.

thank you

@akerouanton
Copy link
Member

We added a "host networking" beta feature to Docker Desktop 4.29. You can find more details on this page: https://docs.docker.com/network/drivers/host/#docker-desktop.

Let me close this ticket as 'won't fix'.

@akerouanton akerouanton closed this as not planned Won't fix, can't repro, duplicate, stale Apr 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests