Mfcttrf

I'm guessing this is a pretty basic question, but I am having a hard time wrapping my head around it.

So my understanding with linear regression, is that it shows how much a change in X, will cause a change in Y. And the same with multiple linear regression.

But can the same be said about logistic regression? What if both of the variables are nominal? Can you do logistic regression this way?

I am currently running an ordinal variable against a nominal one, and I get similar results when I alternate independent vs dependent.
So, my question is should logistic regression be viewed as explaining causal relationships as we do with linear regression? Or is it possible that causality is overwritten by multicollinearity? Leaving us with strictly correlational inferences?
Thanks

Causality has nothing to do with regression. You can regress any variables that are not causally linked. Better way of thinking of regression is "response of Y to X", or "relationship of Y and X". And in this regard it does not matter if the link function is identical (as in the normal regression) or logit function (as in the logistic regression). In logistic regression the response of Y to X will just have different shape due to the logit link function than it would have in normal regression.

So the answer is no, there are no differences in causality.

Neither of them establish causality - unless we talk about specific experimental set-ups (e.g. randomized studies). They show correlation. E.g. a linear model for maximum daytime temperature with the ice cream sales as a predictor cannot show that selling ice cream causes higher temperature, even if higher ice cream sales correlate with higher temperatures. Of course, I could come up with a whole convoluted story of how this works (e.g. all those freezers increase the outdoors temperature in order to produce ice cream), but that would all be post-hoc reasoning. In my example, we can of course immediately tell that I probably got my causality the wrong way around, but in many practical examples that is a lot less clear.

Just the same thing is happening, if I do a logistic regression for "Was it a very hot day" (yes/no).

A causal relationship is defined by a structural model that relates an outcome to its causes. Regression is one way of estimating the parameters of the structural causal model (there are other ways). If the structural model takes the form of a logistic regression model, then a logistic regression model is one way of recovering the true causal parameter. If the structural model takes the form of a linear regression model, then a linear regression model is one way of recovering the true causal parameter. With a binary outcome, a logistic regression model makes more sense because the the output of a logistic regression model can be interpreted as a probability, and part of the outcome-generating process may involve drawing a 0 or 1 with a given probability. With a continuous outcome, a linear regression may make more sense. The choice of model you use should depend on the form of the structural causal model you are trying to approximate. (Note that both linear and logistic regression can be used for both binary and continuous outcomes.)

What many of the posters here are arguing is that regression (logistic or linear) are not inherently causal methods; they can be used to extract a causal parameter from data, but they can be used for other purposes, too. My view is that as long as the predictor precedes the outcome temporally, regression is inherently causal, regardless of your study design. The difference is that the parameter you may estimate from your model may be a biased estimate of the causal parameter. The degree of bias depends on qualities of your design (e.g., whether you have randomized your treatment, whether you are implicitly conditioning on a consequent of treatment, whether you have collected enough variables to remove confounding). Including covariates in a linear or logistic regression is one way to attempt to remove the bias of a causal effect estimate.

When the structural causal model is perfectly reproduced by the model you specify, the causal parameter of interest will be estimated without bias. This tends not to be the case, and so estimating causal parameters using regression can leave you with a biased estimate (if bias is induced by your design). It is for this reason that other answers have been strong in claiming regression is not an inherently causal method; in a bias-inducing design, it's almost impossible to specify a regression model that correctly reproduces the structural causal model that underlies the data.

A final note: people often say correlation does not imply causation, but correlation without confounding does imply causation. Regression is one way to remove confounding. The effectiveness of logistic regression or linear regression at doing so depends on generally unknown qualities of the data-generating structural causal model.